Chart Depicting Lilly's Responses

	Responses During Training
Day		Responses
Magazine Training		3
Shaping Day 1		9
Shaping Day 2		17
Shaping Day 3		39
Shaping Day 4		72
FR2		134
FR3		285
FR5		469
FR7		286
FR7		183
FR10		744
FR12		970
FR15		615
FR20		681
Extinction Day 1		175.6
Extinction Day 2		48.75

Lilly's Weight Chart

	Weight Chart
	Target Weight: 215 grams
Date	Weight	Food
9/12/2012	247 grams	2.6 grams
13-Sep	233	2.3
14-Sep	220	2.4
15-Sep	222	3.3
16-Sep	219	3.1
17-Sep	210	4
18-Sep	205	6.8
19-Sep	210	5.6
20-Sep	216	5.7
21-Sep	211	9.3
22-Sep	210	13.9
23-Sep	229	3
24-Sep	215	6.6
25-Sep	225	3.3
26-Sep	219	3.4
27-Sep	217	3.3
28-Sep	210	3.5
29-Sep	209	4.4
30-Sep	212	4.6
1-Oct	217	3.9
2-Oct	212	4.2
3-Oct	210	4.3
4-Oct	214	Free Feed

Extinction: Sometimes Life is Unfair

   Today was the second day of extinction for Lilly, and I'll tell you this: she was NOT a fan. The goal of the second day was to drastically reduce the number of times Lilly actually pressed the bar, because she had learned that engaging in bar pressing would no longer provide rewards. On the first day of extinction, Lilly still consistently tried to press the bar, although it was at a much lower rate. By the time we got to our second session, you can see from the graph that she had drastically reduced her bar-pressing.
  When Lilly first understood that she wasn't going to get anymore treats, she began exhibiting some really interesting behaviors. First, she started pressing the bar like her life depended on it, which was an excellent example of an extinction burst. I could see that she was frustrated, and her behavior mirrored that frustration. After Lilly attempted to press the bar so many times, she took a little break and started biting on the plastic front of the cage, which was a behavior I hadn't observed in her before. She also started jumping at the light over the bar, as if she thought it would give her a treat if she did something to it. In addition, the number of times she pressed the bar decreased significantly as the session went on. After about 25 minutes of the session, she had almost completely stopped pressing the bar. Today, she's only pressed the bar 51 times in eight minutes (so far) which is a huge decline from her earlier records. I eventually terminated the training session after 23 minutes, because she hadn't pressed the bar in quite some time. I think Lilly might be getting the message- no more treats for her! Although I'm sure she didn't like it, I enjoyed watching her try a variety of behaviors to receive treats again. The graph of responses demonstrates her drastic decline in lever-pressing behavior, and you can see how much her behavior changed, especially on the second day. Sometimes life is rough for a rat!

	Extinction
Extinction Day 1
	5 Minutes	98
	10 Minutes	165
	15 Minutes	194
	20 Minutes	198
	25 Minutes	223
Extinction Day 2
	5 Minutes	42
	10 Minutes	51
	15 Minutes	51
	20 Minutes	51

Lilly's FR Schedule(s)

   Lilly and I spent a great deal of time on the FR schedule. Once we reached the FR10 stage, I decided that I wanted to continue to go upwards. The highest schedule we attained was the FR20 schedule, and I was so proud of Lilly for hanging in that long! I enjoyed going up to a higher level, because it really demonstrated the concept of stretching the ratio. We discussed it in class, but it wasn't until I actually did it myself that I really understood it. She consistently pressed the bar over 600 times at every session once we started the FR12 schedule, which was really cool to watch. You can see in the chart depicting response times (which is in another blog post) to see how much she pressed the bar, and see how the higher up we got in the FR sessions, the more Lilly pressed the bar. She also became more consistent in how often she pressed the bar. For example, when we first started the sessions Lilly was very hesitant and tentative about pressing the bar, even if I manually rewarded her to increase her interest. But as we got further along, she put a lot of time into pressing it, and would press it multiple times in a row until she received a reward. It seemed to me that she had learned it would take more effort to get a treat, and that made her put more work into pressing the bar. However, the fact that we went up to an FR20 schedule made extinction reeeeeeeally difficult when the time came... but more on that later! During our FR sessions, Lilly was about 5 grams underweight, and I theorize this might have contributed to her excessive motivation to press the bar. Either way, as you'll see in the chart documenting her average responses, Lilly was always eager to engage in pressing the bar. The goal of these FR sessions was to increase the number of times Lilly pressed the bar, and I firmly believe we achieved that goal.

Lilly and the Lever: Some Challenges

   So, training Lilly wasn't all just fun and games.We encountered one major problem with pressing the lever, and I had to spend an extra day shaping her so I could correct the issue.
   What's the problem, you ask? Well, instead of using her little paws to press the bar, Lilly somehow started using her neck instead!! If you think that sounds a little weird, let me tell you this: it was. I'm not entirely sure how it got started, but I decided that the best way to handle the problem was to put the computer program on manual reinforcement, and only provide a reward when she pressed the bar with her paws. Although it didn't completely eliminate the behavior, it did reduce it. After about 10 minutes of me only reinforcing presses with her paws, Lilly understood the behavior I wanted from her. This was proven when during the next training session, when I placed her on an FR2 schedule and she only hit the lever with her paws. Although this was a challenge to our training, I was pleased that we overcame it! I'm sure we'll face other difficulties during training, but it was very rewarding to jump over this hurdle.

Lilly Learns About Shaping

  Before I could place Lilly on a fixed reinforcement schedule, I had to shape her to press the bar with her paws. The first goal of shaping was to reward Lilly when she came close to the bar. I programmed the computer to allow me to reward manually and also reward when the bar was pressed, just in case she happened to press it during shaping. Lucky for me, Lilly is an incredibly energetic and curious little creature, so she performed a wide variety of behaviors that were easy for me to reward.
  First, I began by shaping her to come close to the bar. This was difficult because Lilly moved around the box so much that sometimes I didn't catch her soon enough, and I felt that she wasn't understanding the reinforcement. Finally, after several minutes, she began to understand, and I could then began rewarding her when her head was over the lever. Once again this proved difficult, because Lily's constant movement meant I didn't always catch her as soon as she performed the behavior. Because this was so hard to catch, I decided to reward her when she put her nose on the bar, which was a behavior she seemed to be performing often. After I shaped her to place her head above the bar, I decided to end the training session for that day. Lilly seemed preoccupied with sniffing the cage, and I concluded that she was too distracted to move on. We trained for 25 minutes on the first day of shaping, and her weight was 230 grams.
  On the second day, I was able to reinforce not only Lilly placing her nose on the bar, but also placing a paw on the bar. She caught on quickly, but the hardest part was getting her to press down. I decided that the best way to get her to begin hitting the bar was to stop reinforcing her paw placement. I theorized that this would frustrate her, and she would try another behavior. And guess what: I was right!! Lilly was immediately displeased that she couldn't get a reward, and she started pressing down on the bar. We trained for 23 minutes that day, and her weight was 220 grams.
   Day Three of shaping was the moment of truth: did Lilly understand the connection between pressing the bar and getting a reward? Unfortunately, I felt like she did not. Sometimes she would press the bar and go retrieve the reward, and other times she didn't. To compound issues, I was sick and had to miss out on training her for two days in a row, and it was obvious that she had regressed. This meant I had to spend a fourth day of shaping her, to ensure that Lilly had truly learned how to correctly press the bar, AND I had to correct the issue of her using her neck instead of her paws to press down. But as you can see from the video of her final day of shaping, it looked as if Lilly finally understood the connection: if you hit the bar, you'll receive a reward. This meant we could finally move on to FR schedules! On the third day, we trained for 26 minutes and Lilly weighed 222 grams. On the fourth day, we trained for 21 minutes and Lilly weighed 219 grams.

Lilly's second day of shaping:
http://www.youtube.com/watch?v=ucdJ5ODF8GA&feature=youtu.be

Lilly's last day of shaping:
http://www.youtube.com/watch?v=ahKn3cg0dYQ&feature=youtu.be
   - She's still a littleeee confused about the food hopper, but I determined that she had learned the association well enough to move on.

Magazine Training: Lilly Learns About the Food Hopper

   Magazine training is perhaps the most important first step in training your rat to press the lever. The goal of magazine training Lilly was to teach her about the relationship between the light/sound of the food hopper and receiving a treat (which would later become her reward for pressing the lever). Lilly was deprived to 85% of her body weight, to ensure that she would be hungry enough to actually eat the sugar pellets when they were rewarded to her. Her initial starting weight was 247 grams, and the goal weight was 215 grams. By the time I began magazine training, she weighed 233 grams. Our sessions started at 3:30PM every day. In order to magazine shape her, I used the manual reinforcement setting on the computer program. This allowed me to reward her when she put her head near the food hopper, thus creating a relationship between the hopper and the treat. After that, I'd let Lilly walk away, and when she put her head in the hopper again, I'd reward her. We trained for 22 minutes, and I rewarded her 34 times. I would press the button when she was away from the hopper, and upon hearing the noise, Lilly would immediately run over to it. After the session was over, she had learned the connection between the light/sound and the reward, because she responded to the sound of the food hopper, even if she was on the other side of the operant box. I concluded that I could begin shaping the next day.