So let me leave you with this final thought...

   Now that this experience is over, I almost don't know what to do with myself. I'll admit that  I originally thought this project would be time-consuming and a little boring, which wasn't something I was looking forward to. I was excited about getting a rat (finally, an excuse to get a pet!!), but that was pretty much it.
   But thankfully, this entire project exceeded my expectations entirely. Having an opportunity to train my own living rat was a great way to teach myself the concepts we discussed in class, plus it gave me something to talk about when I give campus tours!! But really- I ended up loving this project. I can't wait to teach Lilly more tricks (I'm thinking maybe an obstacle course?), and I know I'll use these concepts of positive reinforcement and extinction in the real world.
   Overall, there aren't many things I'd change about the project. In fact, I probably wouldn't change anything. I really liked doing this blog instead of having to write an entire paper, especially because I could add pictures and videos. I also liked how our training times were staggered to prevent the rats from getting distracted by the other boxes. Although it was sometimes inconvenient for me to train at 3:30PM everyday, or I didn't feel like training at that time, it helped me a lot to just go to Stephens and get it done. I'd definitely recommend keeping the staggered times!!
  Above all else, this was a project about caring- caring for an animal, and caring enough to train them to do something extraordinary. I definitely have a new respect for the researchers that work with lab rats on a daily basis. I'll miss all the fun I had with Lilly, but I can't wait to teach her new tricks in the future!

Lilly vs. Sniffy: Comparing a Live Rat with a Virtual Rat

   Shaping a live rat is very different from shaping a virtual rat. For one thing, the virtual rat doesn't poop everywhere, which is always nice. Also, the time it takes to shape Sniffy was significantly less than shaping Lilly. When working with Sniffy, it took about 60-70 minutes to complete shaping. With Lilly, it took 4 days, or about 120 minutes of training time. That's almost twice as long!!
   In addition, magazine training Sniffy was much easier than magazine training Lilly. It seemed to me that Sniffy quickly understood the rewards from the food hopper, while it took my live rat almost an entire training session to learn the connection. In addition, Sniffy would always go immediately to the food hopper after hearing the sound of the magazine; in training Lilly, she would often wait a few seconds before going to get her treat. I think my biggest issue with Sniffy is that the realism is pretty low. The graphics aren't very good, and the time it takes to train the virtual rat is a lot less than the time it takes to train a real rat. For example, in the Sniffy program you can fast-forward and speed up the training process, which is definitely something that does NOT happen in real life. But other than the realism issue, I generally enjoyed the Sniffy program.
   I think from a learning standpoint, the biggest strength was being able to place Sniffy on a different reinforcement schedule than Lilly. And as I mentioned earlier, I would highly recommend using both the virtual and the live rat for class. I was also able to connect many of the concepts from class to training both rats, and I know I wouldn't have learned as much if I had only trained Sniffy. So although sometimes Sniffy can be time-consuming to work with, I found that using both rats was highly beneficial to my learning.

This is one of Lilly's cumulative records from being on an FR12 schedule. She hit the lever over 900 times that day.

Lilly's Virtual Friend: A Post on Sniffy

   As part of this course, we also had to train a virtual rat named Sniffy. Although he wasn't nearly as exciting as the real thing, it was excellent practice for magazine training and shaping Lilly. The process of magazine training and shaping also went by a lot more quickly with Sniffy than with my live rat. With Sniffy, it only took a little over an hour to shape, while it took several days to shape Lilly. In addition, after finishing the shaping of Sniffy, I decided to put it on a VI schedule. This was schedule was obviously different from shaping, especially because Sniffy pressed the bar on a more regular schedule than he did during shaping.
   One positive of working with Sniffy is that I got some good training tips for Lilly. For example, I learned that it's helpful to reward your rat when they rear up over the lever, because that teaches them that they should be over it in order to push down. Also, because I put Lilly on an advanced FR schedule, I put Sniffy on a VI schedule. It was interesting to see the differences between the two schedules, and I enjoyed having the opportunity to see a different kind of training schedule. I would definitely recommend using both the live and virtual rat for future classes- it's a great opportunity to learn new ways of training, and it helps immensely with training the live rat.

This is a picture of Sniffy, after I finished the shaping process. You can see from the graph that Sniffy fully understands the association between sound and food.

Chart Depicting Lilly's Responses

	Responses During Training
Day		Responses
Magazine Training		3
Shaping Day 1		9
Shaping Day 2		17
Shaping Day 3		39
Shaping Day 4		72
FR2		134
FR3		285
FR5		469
FR7		286
FR7		183
FR10		744
FR12		970
FR15		615
FR20		681
Extinction Day 1		175.6
Extinction Day 2		48.75

Lilly's Weight Chart

	Weight Chart
	Target Weight: 215 grams
Date	Weight	Food
9/12/2012	247 grams	2.6 grams
13-Sep	233	2.3
14-Sep	220	2.4
15-Sep	222	3.3
16-Sep	219	3.1
17-Sep	210	4
18-Sep	205	6.8
19-Sep	210	5.6
20-Sep	216	5.7
21-Sep	211	9.3
22-Sep	210	13.9
23-Sep	229	3
24-Sep	215	6.6
25-Sep	225	3.3
26-Sep	219	3.4
27-Sep	217	3.3
28-Sep	210	3.5
29-Sep	209	4.4
30-Sep	212	4.6
1-Oct	217	3.9
2-Oct	212	4.2
3-Oct	210	4.3
4-Oct	214	Free Feed

Extinction: Sometimes Life is Unfair

   Today was the second day of extinction for Lilly, and I'll tell you this: she was NOT a fan. The goal of the second day was to drastically reduce the number of times Lilly actually pressed the bar, because she had learned that engaging in bar pressing would no longer provide rewards. On the first day of extinction, Lilly still consistently tried to press the bar, although it was at a much lower rate. By the time we got to our second session, you can see from the graph that she had drastically reduced her bar-pressing.
  When Lilly first understood that she wasn't going to get anymore treats, she began exhibiting some really interesting behaviors. First, she started pressing the bar like her life depended on it, which was an excellent example of an extinction burst. I could see that she was frustrated, and her behavior mirrored that frustration. After Lilly attempted to press the bar so many times, she took a little break and started biting on the plastic front of the cage, which was a behavior I hadn't observed in her before. She also started jumping at the light over the bar, as if she thought it would give her a treat if she did something to it. In addition, the number of times she pressed the bar decreased significantly as the session went on. After about 25 minutes of the session, she had almost completely stopped pressing the bar. Today, she's only pressed the bar 51 times in eight minutes (so far) which is a huge decline from her earlier records. I eventually terminated the training session after 23 minutes, because she hadn't pressed the bar in quite some time. I think Lilly might be getting the message- no more treats for her! Although I'm sure she didn't like it, I enjoyed watching her try a variety of behaviors to receive treats again. The graph of responses demonstrates her drastic decline in lever-pressing behavior, and you can see how much her behavior changed, especially on the second day. Sometimes life is rough for a rat!

	Extinction
Extinction Day 1
	5 Minutes	98
	10 Minutes	165
	15 Minutes	194
	20 Minutes	198
	25 Minutes	223
Extinction Day 2
	5 Minutes	42
	10 Minutes	51
	15 Minutes	51
	20 Minutes	51

Lilly's FR Schedule(s)

   Lilly and I spent a great deal of time on the FR schedule. Once we reached the FR10 stage, I decided that I wanted to continue to go upwards. The highest schedule we attained was the FR20 schedule, and I was so proud of Lilly for hanging in that long! I enjoyed going up to a higher level, because it really demonstrated the concept of stretching the ratio. We discussed it in class, but it wasn't until I actually did it myself that I really understood it. She consistently pressed the bar over 600 times at every session once we started the FR12 schedule, which was really cool to watch. You can see in the chart depicting response times (which is in another blog post) to see how much she pressed the bar, and see how the higher up we got in the FR sessions, the more Lilly pressed the bar. She also became more consistent in how often she pressed the bar. For example, when we first started the sessions Lilly was very hesitant and tentative about pressing the bar, even if I manually rewarded her to increase her interest. But as we got further along, she put a lot of time into pressing it, and would press it multiple times in a row until she received a reward. It seemed to me that she had learned it would take more effort to get a treat, and that made her put more work into pressing the bar. However, the fact that we went up to an FR20 schedule made extinction reeeeeeeally difficult when the time came... but more on that later! During our FR sessions, Lilly was about 5 grams underweight, and I theorize this might have contributed to her excessive motivation to press the bar. Either way, as you'll see in the chart documenting her average responses, Lilly was always eager to engage in pressing the bar. The goal of these FR sessions was to increase the number of times Lilly pressed the bar, and I firmly believe we achieved that goal.

Lilly and the Lever: Some Challenges

   So, training Lilly wasn't all just fun and games.We encountered one major problem with pressing the lever, and I had to spend an extra day shaping her so I could correct the issue.
   What's the problem, you ask? Well, instead of using her little paws to press the bar, Lilly somehow started using her neck instead!! If you think that sounds a little weird, let me tell you this: it was. I'm not entirely sure how it got started, but I decided that the best way to handle the problem was to put the computer program on manual reinforcement, and only provide a reward when she pressed the bar with her paws. Although it didn't completely eliminate the behavior, it did reduce it. After about 10 minutes of me only reinforcing presses with her paws, Lilly understood the behavior I wanted from her. This was proven when during the next training session, when I placed her on an FR2 schedule and she only hit the lever with her paws. Although this was a challenge to our training, I was pleased that we overcame it! I'm sure we'll face other difficulties during training, but it was very rewarding to jump over this hurdle.

Lilly Learns About Shaping

  Before I could place Lilly on a fixed reinforcement schedule, I had to shape her to press the bar with her paws. The first goal of shaping was to reward Lilly when she came close to the bar. I programmed the computer to allow me to reward manually and also reward when the bar was pressed, just in case she happened to press it during shaping. Lucky for me, Lilly is an incredibly energetic and curious little creature, so she performed a wide variety of behaviors that were easy for me to reward.
  First, I began by shaping her to come close to the bar. This was difficult because Lilly moved around the box so much that sometimes I didn't catch her soon enough, and I felt that she wasn't understanding the reinforcement. Finally, after several minutes, she began to understand, and I could then began rewarding her when her head was over the lever. Once again this proved difficult, because Lily's constant movement meant I didn't always catch her as soon as she performed the behavior. Because this was so hard to catch, I decided to reward her when she put her nose on the bar, which was a behavior she seemed to be performing often. After I shaped her to place her head above the bar, I decided to end the training session for that day. Lilly seemed preoccupied with sniffing the cage, and I concluded that she was too distracted to move on. We trained for 25 minutes on the first day of shaping, and her weight was 230 grams.
  On the second day, I was able to reinforce not only Lilly placing her nose on the bar, but also placing a paw on the bar. She caught on quickly, but the hardest part was getting her to press down. I decided that the best way to get her to begin hitting the bar was to stop reinforcing her paw placement. I theorized that this would frustrate her, and she would try another behavior. And guess what: I was right!! Lilly was immediately displeased that she couldn't get a reward, and she started pressing down on the bar. We trained for 23 minutes that day, and her weight was 220 grams.
   Day Three of shaping was the moment of truth: did Lilly understand the connection between pressing the bar and getting a reward? Unfortunately, I felt like she did not. Sometimes she would press the bar and go retrieve the reward, and other times she didn't. To compound issues, I was sick and had to miss out on training her for two days in a row, and it was obvious that she had regressed. This meant I had to spend a fourth day of shaping her, to ensure that Lilly had truly learned how to correctly press the bar, AND I had to correct the issue of her using her neck instead of her paws to press down. But as you can see from the video of her final day of shaping, it looked as if Lilly finally understood the connection: if you hit the bar, you'll receive a reward. This meant we could finally move on to FR schedules! On the third day, we trained for 26 minutes and Lilly weighed 222 grams. On the fourth day, we trained for 21 minutes and Lilly weighed 219 grams.

Lilly's second day of shaping:
http://www.youtube.com/watch?v=ucdJ5ODF8GA&feature=youtu.be

Lilly's last day of shaping:
http://www.youtube.com/watch?v=ahKn3cg0dYQ&feature=youtu.be
   - She's still a littleeee confused about the food hopper, but I determined that she had learned the association well enough to move on.

Magazine Training: Lilly Learns About the Food Hopper

   Magazine training is perhaps the most important first step in training your rat to press the lever. The goal of magazine training Lilly was to teach her about the relationship between the light/sound of the food hopper and receiving a treat (which would later become her reward for pressing the lever). Lilly was deprived to 85% of her body weight, to ensure that she would be hungry enough to actually eat the sugar pellets when they were rewarded to her. Her initial starting weight was 247 grams, and the goal weight was 215 grams. By the time I began magazine training, she weighed 233 grams. Our sessions started at 3:30PM every day. In order to magazine shape her, I used the manual reinforcement setting on the computer program. This allowed me to reward her when she put her head near the food hopper, thus creating a relationship between the hopper and the treat. After that, I'd let Lilly walk away, and when she put her head in the hopper again, I'd reward her. We trained for 22 minutes, and I rewarded her 34 times. I would press the button when she was away from the hopper, and upon hearing the noise, Lilly would immediately run over to it. After the session was over, she had learned the connection between the light/sound and the reward, because she responded to the sound of the food hopper, even if she was on the other side of the operant box. I concluded that I could begin shaping the next day.