If you’ve been reading my blog, by now you know that Barnum and I are hard at work on Sue Ailsby’s Training Levels. In addition to mastering a set of behaviors, the trainer/handler has a written homework assignment for each level. Here is Level Two’s question and my answer, which I have livened up with some Barnum photos. Shortly I hope to put up a page with short videos of Barnum and me getting tested on eight of the L2 behaviors. (There are 16 behaviors to master in L2.)
Handler describes, in writing, the four “legs” of operant conditioning, and the definition of “reinforcement” and “punishment.”
The first leg is positive reinforcement.
On the training and behaviorist lists, abbreviated “R+.” Positive reinforcement is giving something to the learner (adding something to the learner’s environment) that the learner needs or wants. A positive reinforcer makes the behavior more likely to happen again because the learner wants to get that reward again.
Note: ALL forms of reinforcements and punishments are intended to alter future behavior, however, their success varies depending on which “leg” is used and also on other factors, especially whether the reward or aversive is effective/relevant/appropriate to that particular dog/learner, and also whether the timing makes it clear what is being punished/rewarded.
For Barnum, R+ are food, play with other dogs, playing tug, being let outside, chasing a ball, praise, belly rubs when he is lying on his back, walks, being let off leash to run, or being allowed to dig in soft dirt or climb into the tub when it’s only partly full.
The nice thing about R+ is that there is no “fallout.” If you screw up and accidentally reward when you didn’t mean to, or miss an opportunity to reward, it might create or strengthen an undesirable behavior that will have to be extinguished, but it doesn’t tend to cause distress for the learner, inhibit further learning, damage trust, etc.
I will use the same three real-life examples of applying these forms of reinforcement or punishment (sometimes intentional, sometimes unintentional) with Barnum for each of the three legs.
Examples of Positive Reinforcement
Behavior: Jumping on my bed. (Undesirable)
R+ responses:
- Click/treat (c/t) for getting off the bed (four on the floor) with hand and/or voice cue or lure of food treat.
- C/t for standing, sitting, lying next to bed. C/t for going into kennel. C/t with stuffed Kong or meal for lying quietly in crate or next to bed for extended period instead of jumping on bed.
- Barnum jumps onto bed and won’t get off, so I get up and leave as an attempt at positive punishment, forgetting that I left chicken salad on bed; Barnum eats chicken and is positively reinforced for being on bed. (Oops!)
- Being on the bed seems to be both self-reinforcing as well as a means to an end: Getting on the bed brings attention (I stop eating or working at computer to get him off), food (see above), closeness to me, and also seems to be enjoyable in itself as a soft, high location. (Nevermind that he has four dog beds of his own, Barnum still wants to get on my bed or the couch, even when I’m not on them.)
- Since it’s impossible to launch a clicker session every time Barnum’s around my bed (that would be all the time except if we are out playing, training, or going for a walk), I try to notice when he seems like he is about to jump or wants to jump, and then I praise, play, or c/t him for having “four on the floor” and try to initiate a game or give him something (appropriate!) to chew.
Behavior: Walking on a loose leash. (Desirable.)
Why is pulling on leash undesirable? Exhibit A:
"There's Lucy! I want to play with her! I'm coming, Lucy!" CRACK! Ka-PING!
R+ techniques for getting a loose leash:
- C/t any time leash is loose.
- Praise when leash is loose.
- Walk toward desirable food or other source of pleasure (another dog) when leash is loose.
- Offer a walk or game as reward for loose leash.
Behavior: Letting himself out by jumping against the screen door. Desirability: It’s complicated.
While it was never my goal for Barnum to learn this neat trick (ahem), it was more desirable to have Barnum let himself out than for him to have accidents inside — when we were still doing toilet training — since I have not always been good at reading his (to me) often subtle indicators that he needs to go out.
However, now he has bladder/bowel control, and we have better communication, so it’s not necessary. It’s bad for the door and screen, especially now that he’s so big and strong and doing damage; it’s bad for our communication and working relationship (too much freedom without earning it); and when winter comes and he can’t let himself out that way, I don’t want him to have accidents inside again (although I don’t think he will because toileting outside has become so ingrained, and we have worked out other options).
This behavior was born when Barnum one day scratched on the door to indicate, “I want out,” and the door swung open. That was very reinforcing: the power! The control! He learned that if he exerts enough force (jumping up with both legs and really putting his weight into it works almost every time!), he didn’t need to wait for me. Having the ability to let himself out at will is a very powerful reinforcer, not just for offering bladder/bowel relief but for playing in the yard, greeting people coming in the gate, freedom, change of scene, etc. This behavior is self-reinforcing.
R+ I’m using to change the situation:
- Don’t allow him access to the screen door, thus cutting off the self-reinforcement of him being able to let himself out.
- C/t him for any behavior other than jumping/swiping at the screen door (when the screen door is available to him).
- Trained him to ring a bell (by targeting the bell with his nose and earning c/t for successively harder nudges till he is ringing it loudly enough to be heard) which I then hung by the door. Whenever he rings the bell, I ask him if he wants to go out (R+); I toss the treats outside for him to eat (R+); and I let him out (R+), even if I know he doesn’t need to eliminate. (At this point, I’m just working on communicating, “If you want to go out, slamming into the door won’t work, but ringing the bell will.”) It’s a long process because any time anyone (my four PCAs, my partner, or I), forget and allow him access to the screen door, there is the possibility of him letting himself out, thus putting that behavior on a variable schedule of reinforcement, making it über hard to extinguish.
The second leg is positive punishment.
I assume this is abbreviated P+, though I have not seen it on the lists. Positive punishment is adding something to the learner’s environment that s/he dislikes in order to stop an undesirable behavior. P+ might make behavior less likely to occur again because the learner wants to avoid the punisher.
Positive punishment has the most likelihood of causing fallout. Not only is it always unpleasant for the learner, it is often also unpleasant for the trainer (although, if people were honest about it, a lot of the time it is really just venting anger or frustration on the dog in the name of changing behavior, when it is actually more like revenge). The learner often associates the aversive experience with the trainer, which might teach the learner to avoid the trainer or to only do the behavior in the trainer’s absence, as opposed to extinguishing the behavior altogether. It’s also hard to time correctly, and an ill-timed punishment often creates more problems than it solves, as it can punish a desirable behavior that happens after the undesirable one has ceased.
Examples of Positive Punishment
Behavior: Jumping on my bed. (Undesirable.)
"I see no reason at all why I shouldn't be allowed on your bed whenever I want. What's the prob, dude?"
- Yell at him for jumping on bed. (Doesn’t work, he doesn’t care and is often glad of the attention. Potential fallout: Barnum learns to dislike being around me or learns to tune me out.)
- Try to push him off the bed. (Doesn’t work, he thinks it’s a game, like tug, and can therefore be R+ instead!)
- Get up and leaving the room. (Works if his goal in jumping up was being closer to me or getting attention; doesn’t work if he just wants to be on the bed to get a better view out the window or a change of pace or to get at my dinner.) He also doesn’t seem to learn from this.
Behavior: Walking on a loose leash. (Desirable.)
Positive punisher: Accidentally run over his toes when he is walking very close to the chair, giving me good eye contact, and we are both so focused on clicks and treats that we lose track of his paws and my wheels. (Aagh!) The result is that, until I counteract this mistake, he walks farther from my chair, hangs back instead of walking next to me, and enjoys himself less. Not intentional!
The traditional punisher for pulling on leash is using choke chains, prong collars, etc., when the dog pulls, but I don’t do that.
Behavior: Letting himself out by jumping against the screen door. (Undesirable.)
Positive punisher: Mildly scold (Eh!) him when he jumps on door. (Sometimes works if I catch his attention and he stops before he goes out, but usually I am too late — there’s that timing thing again!)
The third leg is negative reinforcement.
Abbreviated R-. Negative reinforcement is removing something from the environment that the learner dislikes (finds aversive) in order to make the event that comes with the removal more likely to occur again. Although negative reinforcement is not the same as punishment, using R- necessarily requires an aversive. In other words, until the aversive is removed to reward the change in behavior, whatever came before was being punished.
Examples of Negative Reinforcement
Behavior: Jumping on my bed. (Undesirable.)
- While Barnum is on my bed, I silently turn my back to him, giving no eye contact. When he gets off the bed, I shower him with praise, petting, c/t. (The aversive is me ignoring him, the removal of it = negative reinforcement/R-.)
- I make grumbling/growling noises when he’s on the bed. When he jumps off, I smile and stop grumbling. (Grumbling = aversive; stopping grumbling = R-.) This worked the first two times I tried, and then he realized I was silly and no actual threat, thought it was a great game; in other words, not effective because it did not alter the behavior and very briefly, was actually probably R+!)
- I get up and leave the room, shutting the door behind me. Barnum is alone in my bedroom. (He doesn’t like to be apart from me — or alone in general. Social isolation is usually a big punisher for dogs.) When he jumps off the bed, I come back into the room, smiling warmly, and give him eye contact. (Social isolation = aversive; ending social isolation = R-.)
Behavior: Walking on a loose leash. (Desirable.)
Why is pulling on lead undesirable? Exhibit B:
"I want to get to the pond NOW!" Pffft-BOING! "Ah, that's better!"
Any time Barnum pulls on lead, I walk backwards from whatever is desirable that he’s pulling toward (food, another dog, an interesting smell). When he comes back to my side, and the leash goes slack, I stop pulling him away from the desirable smell/object/dog and begin walking toward it again. (The aversive is having to retreat from desirable smell/dog/food; discontinuing the march backwards = R-.)
Behavior: Letting himself out by jumping against the screen door. (Undesirable.)
Don’t let him out when he flings himself against the door:
- Hold the door closed.
- Block the screen door with the winter door.
- Put him on a leash that’s too short for him to get outside even if he manages to open the door.)
All of these efforts are aversive because they are frustrating his desire to 1. have control over going out, 2. go out and play, and/or 3. relieve himself. When he stops trying to slam the screen door and/or rings the bell instead, I give him access to the outdoors. (The aversive is inability go out; the removal of that restriction is R-.)
The fourth leg is negative punishment.
Again, I presume this is abbreviated P-, although I don’t know if I’ve ever seen it discussed outside of theoretical discussions. (Since I’m on R+ lists and read R+ blogs, that’s mostly what I hear about.) Negative punishment is taking away something the learner wants/needs in order to reinforce the opposite of that behavior.
Behavior: Jumping on my bed. (Undesirable.)
P- responses:
- Turning my back/looking away/ignoring him when he is on the bed. (This actually does work, though slowly, in combination with R+ when his feet hit the floor.)
- Can’t think of anything else except maybe moving any food away from him so he doesn’t get to sniff and/or eat it, but I try not to let that happen in the first place.
Behavior: Walking on a loose leash. (Desirable.)
Any time Barnum pulls on the leash, I back away from our focal point, which is something he wants: A bowl of pungent food, another dog, something he wants to sniff. This is slow going, but combined with R+ of c/t when leash is slack and of eventually getting to the object of his desire, it’s effective.
Behavior: Letting himself out by jumping against the screen door. (Undesirable.)
Eliminating access to the yard has been an effective P-, in combination with R+ for ringing the bell to indicate he wants to go out instead of slamming into the door.
"Lying in the grass feels soooo good! Why wouldn't I want to get out here and enjoy it whenever the whim strikes me?"
* * *
Oy! That felt like I was back in psychology class, taking an exam! My brain hurts! Also, a lot of these overlap so much that some of it seems to be engaging in a bit of (as my college academic advisor once shocked me by saying), “mental masturbation.”
This cognitively impaired, hard-trainin’ chick needs a rest!
However, we have tested and passed and videotaped(!) several of the L2 behaviors. I just need to do the closed captioning and transcripts, and I will post those, too. Then you can see the practice, and not read so much theory.
After Gadget 