It's been a disappointing evening at the Cosby household this Halloween. We've only had two trick-or-treaters; they were appropriately grateful, but still. This has left me a lot of time to do the grading I didn't do yesterday, because I was doing schoolwork. It's also left me a lot of time to do the lesson plans I didn't do earlier today, because I was busy making calabaza en tacha for Día de los Muertos. La Señora Cosby was once again gracious enough to make calaveras de azúcar, for my not-officially-a-Spanish-club to decorate.
So, of course, I'm blogging.
I actually tripped across another book I want, and I wanted to record it here before I forgot it.
O'Connor, K. (2009). How to grade for learning. Linking grades to standards. 3d ed. Corwin. ISBN 9781412953825.
O'Connor signed off on PowerSchools, which is the grading software we're using. I sort of dropped the ball on the whole standards-based grading thing this year, so it's much more traditional-grading than I would like. For what it's worth, I'm making sure that the standards assessments are worth four times as much as the practices and homework and whatnot. So, there.
Happy Halloween, everybody.
Sunday, October 31, 2010
Saturday, October 23, 2010
Value-added analysis of auto mechanics
KALAMAZOO AUTO MECHANIC RATINGS
About 1,000 Kalamazoo auto mechanics and 300 auto body shops are included in the Señor Cosby’s database of “value-added” ratings.
Domestic and imported vehicles’ mechanics who worked on at least 10 automobiles from 2003 to 2009 were evaluated by the Blog analysis. Most of Kalamazoo’s auto mechanics are included. Evaluations for independent, front-yard auto mechanics that do not report to the state authority were not available.
An auto mechanic’s value-added rating is based on a few key indicators of performance of his or her customers’ vehicles. The difference between expected performance and actual performance is the “value” a mechanic added or subtracted during the year. A body shop’s value-added rating is based on the performance of all cars worked on during that period.
Small differences in rankings are not statistically significant, particularly for those rated near the average. In some cases, recent gains made by mechanics and body shops may not be reflected in their ratings.
Although value-added measures do not capture everything that goes into making a good mechanic or school, the Blog decided to make the ratings available because the bear on the performance of providers of important services, and in the belief that auto owners and the public have a right to the information.
FAQ:
What is “value-added” analysis?
“Value added” analysis is a statistical method that estimates the effectiveness of a mechanic or shop by looking at key indicators of performance of the cars they’ve worked on--in this instance, the functioning of the left turn signal, the fuel efficiency measured in kilometers per hour, and the acceleration rate, measured as the time taken to go from 0 to 100 kilometers per hour. These scores are aggregated to form a single rating. Past scores are used to project each car’s future performance. The difference between the car’s actual performance and projected results is the estimated “value” that the mechanic or shop added (or subtracted) during the year.
Do value-added scores tell you everything you need to know about a mechanic or a body shop?
Not at all. Even advocates of the method say it should count for half or less of a mechanic’s overall evaluation. In reviewing a mechanic’s performance, administrators may want to consider their supervisor’s observations, the quality of the automobile’s appearance, mechanics’ abilities to work on systems other than the three mentioned, and many other factors. Similarly, auto owners looking for a shop for their car may also want to consider factors such as the shop’s state credentials, honesty and integrity policies, and their own impressions of the mechanics and the shop.
Why publish individual mechanic’s ratings?
Research has repeatedly found that mechanics are the single most shop-related factor in an auto’s long-term functioning, yet until now, auto owners have had no objective information about their effectiveness. The Kalamazoo Auto Body Workers Association has had the underlying data in hand for years but has not used them to inform auto owners--or mechanics themselves--about how mechanics are doing. The Blog made the decision to release the information because it bears on the performance of service providers and the belief that auto owners and the public have a right to judge it for themselves.
How accurate are mechanics’ rankings?
Value-added scores are estimates, not precise measures, and readers should not place too much emphasis on small differences in mechanics’ percentiles. As a technical matter, both sampling error and measurement error contribute to the variability of the estimated mechanic effects. The percentile rank is based on a point estimate for each mechanic, but the mechanic’s "true" rank falls in a range around each point estimate. In general, the potential for error is smaller at the high and low end of the scale and wider in the middle. Put another way, the scores are most accurate for the most effective and least effective mechanics and somewhat less so for mechanics whose scores are closer to average. The range of potential values for acceleration was plus or minus 5 percentile points at the 20th and 80th percentiles. For turn signals and fuel efficiency it was plus or minus 7 at those percentiles.
Is a mechanic’s or schools score affected by low-achieving vehicles, diesel trucks, Fords, or other cars with challenges?
Generally not. By comparing each vehicle’s results with its past performance, value-added largely controls for such differences, leveling the playing field among mechanics and shops. This distinction means that the assessments of the auto mechanics are strictly measures of an individual mechanic’s ability to exceed expectations. Research using Kalamazoo Auto Associates data has found that teachers with a high percentage of autos who are easily-repaired autos or Fords have no meaningful advantage or disadvantage under the value-added approach. The same applies to mechanics with high numbers of Audis, at one end, or Kias, at the other.
LA Times Teacher Ratings
LA Times value-added FAQ
About 1,000 Kalamazoo auto mechanics and 300 auto body shops are included in the Señor Cosby’s database of “value-added” ratings.
Domestic and imported vehicles’ mechanics who worked on at least 10 automobiles from 2003 to 2009 were evaluated by the Blog analysis. Most of Kalamazoo’s auto mechanics are included. Evaluations for independent, front-yard auto mechanics that do not report to the state authority were not available.
An auto mechanic’s value-added rating is based on a few key indicators of performance of his or her customers’ vehicles. The difference between expected performance and actual performance is the “value” a mechanic added or subtracted during the year. A body shop’s value-added rating is based on the performance of all cars worked on during that period.
Small differences in rankings are not statistically significant, particularly for those rated near the average. In some cases, recent gains made by mechanics and body shops may not be reflected in their ratings.
Although value-added measures do not capture everything that goes into making a good mechanic or school, the Blog decided to make the ratings available because the bear on the performance of providers of important services, and in the belief that auto owners and the public have a right to the information.
FAQ:
What is “value-added” analysis?
“Value added” analysis is a statistical method that estimates the effectiveness of a mechanic or shop by looking at key indicators of performance of the cars they’ve worked on--in this instance, the functioning of the left turn signal, the fuel efficiency measured in kilometers per hour, and the acceleration rate, measured as the time taken to go from 0 to 100 kilometers per hour. These scores are aggregated to form a single rating. Past scores are used to project each car’s future performance. The difference between the car’s actual performance and projected results is the estimated “value” that the mechanic or shop added (or subtracted) during the year.
Do value-added scores tell you everything you need to know about a mechanic or a body shop?
Not at all. Even advocates of the method say it should count for half or less of a mechanic’s overall evaluation. In reviewing a mechanic’s performance, administrators may want to consider their supervisor’s observations, the quality of the automobile’s appearance, mechanics’ abilities to work on systems other than the three mentioned, and many other factors. Similarly, auto owners looking for a shop for their car may also want to consider factors such as the shop’s state credentials, honesty and integrity policies, and their own impressions of the mechanics and the shop.
Why publish individual mechanic’s ratings?
Research has repeatedly found that mechanics are the single most shop-related factor in an auto’s long-term functioning, yet until now, auto owners have had no objective information about their effectiveness. The Kalamazoo Auto Body Workers Association has had the underlying data in hand for years but has not used them to inform auto owners--or mechanics themselves--about how mechanics are doing. The Blog made the decision to release the information because it bears on the performance of service providers and the belief that auto owners and the public have a right to judge it for themselves.
How accurate are mechanics’ rankings?
Value-added scores are estimates, not precise measures, and readers should not place too much emphasis on small differences in mechanics’ percentiles. As a technical matter, both sampling error and measurement error contribute to the variability of the estimated mechanic effects. The percentile rank is based on a point estimate for each mechanic, but the mechanic’s "true" rank falls in a range around each point estimate. In general, the potential for error is smaller at the high and low end of the scale and wider in the middle. Put another way, the scores are most accurate for the most effective and least effective mechanics and somewhat less so for mechanics whose scores are closer to average. The range of potential values for acceleration was plus or minus 5 percentile points at the 20th and 80th percentiles. For turn signals and fuel efficiency it was plus or minus 7 at those percentiles.
Is a mechanic’s or schools score affected by low-achieving vehicles, diesel trucks, Fords, or other cars with challenges?
Generally not. By comparing each vehicle’s results with its past performance, value-added largely controls for such differences, leveling the playing field among mechanics and shops. This distinction means that the assessments of the auto mechanics are strictly measures of an individual mechanic’s ability to exceed expectations. Research using Kalamazoo Auto Associates data has found that teachers with a high percentage of autos who are easily-repaired autos or Fords have no meaningful advantage or disadvantage under the value-added approach. The same applies to mechanics with high numbers of Audis, at one end, or Kias, at the other.
LA Times Teacher Ratings
LA Times value-added FAQ
Thursday, October 21, 2010
MiWLA
I went to the MiWLA conference today. I always like going to this conference; it reaffirms my sense of competence as a teacher, and I always come away with a few good ideas. In the past, I often went with a sense of insecurity in my job, and came away feeling reaffirmed.
Today, I went feeling great in my job, and came away wanting to be anything but a Spanish teacher.
I have been toying with the idea of learning to program in a few different languages, and creating language content. I've especially been excited by the idea of electric textbooks, especially of the open source variety. I talked to a small business owner in the educational content field, and got some great ideas for how to move forward with a similar situation.
I went to a meeting called "K-16 Articulation." Now I want to be an ISD curriculum advisor.
The thing is, I'm still super-excited to go back to my school tomorrow.
Today, I went feeling great in my job, and came away wanting to be anything but a Spanish teacher.
I have been toying with the idea of learning to program in a few different languages, and creating language content. I've especially been excited by the idea of electric textbooks, especially of the open source variety. I talked to a small business owner in the educational content field, and got some great ideas for how to move forward with a similar situation.
I went to a meeting called "K-16 Articulation." Now I want to be an ISD curriculum advisor.
The thing is, I'm still super-excited to go back to my school tomorrow.
Sunday, October 17, 2010
Animation test
I don't remember if I've already tested this, and I can't be bothered to search the archives to find out. I should actually be working right now.
Go!Animate is an online tool for creating flash animations with relative ease. I say relative because it's still a fairly time-consuming process, and it isn't quite as intuitive as it likes to think it is.
I created this animation because in Spanish I, our most recent unit was about basic socializing. The exam for the unit is a one-on-one conversation. I have one student whom I know knows all the stuff, but I don't think I've ever heard her speak individually in class. My thought was that, instead of having a straight-up conversation with her, she could help me do the voice-over work with this cartoon. It turns out that you can't directly record voice-overs into the video--you have to use a different piece of software to record them, and then import them into the animation. If your objective is creating videos, that's not a problem, but if your objective is getting reluctant students to speak, you add a whole layer of abstraction and hassle that you don't want. So that idea isn't really going to work.
Anyway, without further ado, two methods of sharing the video: the link,
http://goanimate.com/movie/0y_W4EyibzgY?utm_source=linkshare
and the video embedding:
GoAnimate.com: Las presentaciones by jcos
Like it? Create your own at GoAnimate.com. It's free and fun!
Update: You can sort of record your own voice. You call a telephone number, and it will record you, and by some mechanism, it ends up attached to your video. It's still not as good as just pressing a record button and talking to the computer, but it might be able to work.
Go!Animate is an online tool for creating flash animations with relative ease. I say relative because it's still a fairly time-consuming process, and it isn't quite as intuitive as it likes to think it is.
I created this animation because in Spanish I, our most recent unit was about basic socializing. The exam for the unit is a one-on-one conversation. I have one student whom I know knows all the stuff, but I don't think I've ever heard her speak individually in class. My thought was that, instead of having a straight-up conversation with her, she could help me do the voice-over work with this cartoon. It turns out that you can't directly record voice-overs into the video--you have to use a different piece of software to record them, and then import them into the animation. If your objective is creating videos, that's not a problem, but if your objective is getting reluctant students to speak, you add a whole layer of abstraction and hassle that you don't want. So that idea isn't really going to work.
Anyway, without further ado, two methods of sharing the video: the link,
http://goanimate.com/movie/0y_W4EyibzgY?utm_source=linkshare
and the video embedding:
GoAnimate.com: Las presentaciones by jcos
Like it? Create your own at GoAnimate.com. It's free and fun!
Update: You can sort of record your own voice. You call a telephone number, and it will record you, and by some mechanism, it ends up attached to your video. It's still not as good as just pressing a record button and talking to the computer, but it might be able to work.
Teachers' unions
I did a brief spiel on Cesar Chavez for Hispanic Heritage Month, and in it I emphasized the role of the National Farm Workers' Association. My seventh-grade class, which often calls out anything I say which even hints at political overtones, asked me many questions which made it clear they'd heard that unions were bad. My eighth graders just wanted to know what unions did for me.
As I answered their questions, I tried to make it clear to them that I was a biased source of information, that everything I was going to tell them was pro-union, and that therefore I couldn't be trusted to provide the whole story. (It's more important that they know valid information when they hear it than that they like teachers' unions.) But it made me think again about what unions are, what unions do, and why unions are generally unpopular in the US and generally getting less so.
And then I came across this excellent post by Audrey Watters, about why almost everything they say about teachers' unions is wrong, or at least skewed. I would only add the following things.
When teachers' unions oppose reform, it is not because we (the teachers, and the members of the unions) are opposed to improvements. We are, however, opposed to being taken advantage of. Part of what unions do is make sure that their members' working conditions improve over time, or at least not get worse.
And part of the argument that a certain flavor of education "reformer" makes is that teachers do not deliver value for money. The obvious solution to this is for teachers to increase their work load and complexity by an undetermined amount up to infinity without an increase in compensation in other areas. They call it--and this might be the worst part--accountability. (As this paragraph currently exists, it's awfully straw man. I'll find some examples--well, honestly, probably when I retire and write my book in the history of education at the turn of the century.) Unions (at their best) try to slow that process down somewhat, to force consideration of whether each change is going to actually improve the students' education experience, and whether it would place undue hardship on the teachers.
This is the take-away quote from this article: "If unions oppose merit pay, it isn’t because they want to just reward long-time teachers with higher pay, quality of instruction be-damned. It’s because what counts as “merit” in these proposals is not the same as being a good teacher."
As I answered their questions, I tried to make it clear to them that I was a biased source of information, that everything I was going to tell them was pro-union, and that therefore I couldn't be trusted to provide the whole story. (It's more important that they know valid information when they hear it than that they like teachers' unions.) But it made me think again about what unions are, what unions do, and why unions are generally unpopular in the US and generally getting less so.
And then I came across this excellent post by Audrey Watters, about why almost everything they say about teachers' unions is wrong, or at least skewed. I would only add the following things.
When teachers' unions oppose reform, it is not because we (the teachers, and the members of the unions) are opposed to improvements. We are, however, opposed to being taken advantage of. Part of what unions do is make sure that their members' working conditions improve over time, or at least not get worse.
And part of the argument that a certain flavor of education "reformer" makes is that teachers do not deliver value for money. The obvious solution to this is for teachers to increase their work load and complexity by an undetermined amount up to infinity without an increase in compensation in other areas. They call it--and this might be the worst part--accountability. (As this paragraph currently exists, it's awfully straw man. I'll find some examples--well, honestly, probably when I retire and write my book in the history of education at the turn of the century.) Unions (at their best) try to slow that process down somewhat, to force consideration of whether each change is going to actually improve the students' education experience, and whether it would place undue hardship on the teachers.
This is the take-away quote from this article: "If unions oppose merit pay, it isn’t because they want to just reward long-time teachers with higher pay, quality of instruction be-damned. It’s because what counts as “merit” in these proposals is not the same as being a good teacher."
Friday, October 15, 2010
Stealing other peoples' links
Online MP3-editing software? Sweet!
http://cutmp3.net/
I can do it in Garage Band to a certain extent, but unless I want to pass my laptop around the classroom every time I want a student to fiddle with a song, I'm the only one.
Great thanks and hat tip to Larry Ferlazzo for pointing it out.
http://cutmp3.net/
I can do it in Garage Band to a certain extent, but unless I want to pass my laptop around the classroom every time I want a student to fiddle with a song, I'm the only one.
Great thanks and hat tip to Larry Ferlazzo for pointing it out.
Saturday, October 9, 2010
More books I want
The brain that changes itself, by Norman Doidge. Recommended to me by Elaine. It's apparently about the way your brain changes its shape all through your life, and the long-term value of learning. Since what I do is highly dependent on brain research, and since Elaine declares it highly readable, I'll be acquiring a copy as soon as feasible.
Sunday, October 3, 2010
Well, that's September gone, then.
This has been a REALLY good start to the year, I think. I hope my students would agree. I actually spent the first two weeks on community building, and have an outline for continuing it through. I have enforced my rules consistently, and when I noticed a big upsurge in students pushing the boundaries, I responded in what I think is the appropriate way. The students had seen the consequences for consistent disruptive behavior already, so the students who were testing the rules were unsurprised by their consequences. After a few minutes, I found an excuse do to a re-focusing activity, like Braingym. The next day, we re-covered our expectations, a la Randy Sprick, and rehearsed the routine where the students fell apart. This was in my 8th grade Spanish class. In almost none of my other classes have I had any difficulty that I couldn't pin on "the teacher kept us in the chair too long." In fact, in my honors English 12 class, I have asked the students to re-write the rules--none of my original rules are a problem.
All of this almost certainly has more to do with the school climate and the students themselves than with my opening sequence. This school is something of an anomaly. There's no school-wide positive behavior support system, the school expectations are not posted on banners all over the walls. And yet the students know. And even more amazing, they follow them. I am frankly stunned at the internal level of communication between faculty, staff and administration that must be going on to make this happen.
What will be REALLY interesting is to see what happens in December. Our football season ends in September, and the football coach is evidently a key part of keeping this whole system running. We're a "Friday Night Lights" kind of school, except instead of the football players expecting special privileges because of who they are, they're held to a higher standard of comportment and academics. It's awesome.
I'm clearly well-liked by the students. My Spanish students tell their parents they like my class, and their parents tell my principal, and my principal tells me. In return, I call students' homes as often as I can to gush about the wonderful things they're doing. If it keeps up like this, the positive behavior stuff will just end up running everything. And wouldn't that be nice?
My Spanish 7 and 8 classes are what I expected them to be--start with social niceties, and go on from there. (I expected my 8th graders to have had Spanish 7, and evidently this isn't universally true. So that was a surprise.) My Spanish I class is a little different. They all claimed some familiarity with Spanish from middle school, but none of them could tell me what they knew, and nothing I've taught them so far
As for my actual performance in delivering content, I give myself a B in my Spanish classes, and a pretty generous C in English class. In my early Spanish classes, I teach students the geography of the Spanish-speaking world. (It's one of the content expectations. I didn't write them.) It's part of my Schmoozing 101 unit (hat tip to Annette from the County ISD for the name); students have to know where their Spanish-speaking friends are from without running to Google Earth. (Although that is precisely what I do.) But it's the sort of thing I hate--it's basically memorizing, it takes FOREVER to do right, and apart from some commands, I have a hard time doing it in Spanish. So the students aren't speaking as much Spanish as I want them to. (I've finally got the "draw your own map" project doing what I want it to do, though...yay!) So they're learning what I'm teaching, I'm teaching it in the right way, but it doesn't feel like the right thing to teach. I'll be working on that for future classes.
My Spanish II has thrown me a complete curve ball, because they're grammarians at the fourth-year level, but have the communication skills of people who have never spoken Spanish. Because they haven't. They have surprisingly strong packets of vocabulary, but there's no reliable way to predict what they have. I've spent most of the last month selling Spanish as a vehicle for communication, and met with a few days of wide-eyed terror-stricken stares when I refused to translate directions. They're starting to catch on, though, and I think they love it. We started a unit in which we'll review everything that happens in a day, with a focus on them speaking Spanish every single day. After that, it will be my standard Spanish II curriculum, with a de-emphasis on grammatical concepts that they already know extremely well, and with plenty of time for back-filling.
Unsurprisingly, my English classes are a little meh. I have a clear learning goal, and a good vision of what steps we need to take to get there. I think my day-do-day practice is good instruction, because a lot of my students are doing good learning, some of them actually against their will. (That was kind of a joke.) I can clean up some of my routines, particularly the beginning of class. But there's a gap, and I keep feeling like there's something missing. I look around and wonder what I'm not teaching. I may ask Annette from the ISD if she has time to observe me teach someday soon, and give me some pointers.
All of this almost certainly has more to do with the school climate and the students themselves than with my opening sequence. This school is something of an anomaly. There's no school-wide positive behavior support system, the school expectations are not posted on banners all over the walls. And yet the students know. And even more amazing, they follow them. I am frankly stunned at the internal level of communication between faculty, staff and administration that must be going on to make this happen.
What will be REALLY interesting is to see what happens in December. Our football season ends in September, and the football coach is evidently a key part of keeping this whole system running. We're a "Friday Night Lights" kind of school, except instead of the football players expecting special privileges because of who they are, they're held to a higher standard of comportment and academics. It's awesome.
I'm clearly well-liked by the students. My Spanish students tell their parents they like my class, and their parents tell my principal, and my principal tells me. In return, I call students' homes as often as I can to gush about the wonderful things they're doing. If it keeps up like this, the positive behavior stuff will just end up running everything. And wouldn't that be nice?
My Spanish 7 and 8 classes are what I expected them to be--start with social niceties, and go on from there. (I expected my 8th graders to have had Spanish 7, and evidently this isn't universally true. So that was a surprise.) My Spanish I class is a little different. They all claimed some familiarity with Spanish from middle school, but none of them could tell me what they knew, and nothing I've taught them so far
As for my actual performance in delivering content, I give myself a B in my Spanish classes, and a pretty generous C in English class. In my early Spanish classes, I teach students the geography of the Spanish-speaking world. (It's one of the content expectations. I didn't write them.) It's part of my Schmoozing 101 unit (hat tip to Annette from the County ISD for the name); students have to know where their Spanish-speaking friends are from without running to Google Earth. (Although that is precisely what I do.) But it's the sort of thing I hate--it's basically memorizing, it takes FOREVER to do right, and apart from some commands, I have a hard time doing it in Spanish. So the students aren't speaking as much Spanish as I want them to. (I've finally got the "draw your own map" project doing what I want it to do, though...yay!) So they're learning what I'm teaching, I'm teaching it in the right way, but it doesn't feel like the right thing to teach. I'll be working on that for future classes.
My Spanish II has thrown me a complete curve ball, because they're grammarians at the fourth-year level, but have the communication skills of people who have never spoken Spanish. Because they haven't. They have surprisingly strong packets of vocabulary, but there's no reliable way to predict what they have. I've spent most of the last month selling Spanish as a vehicle for communication, and met with a few days of wide-eyed terror-stricken stares when I refused to translate directions. They're starting to catch on, though, and I think they love it. We started a unit in which we'll review everything that happens in a day, with a focus on them speaking Spanish every single day. After that, it will be my standard Spanish II curriculum, with a de-emphasis on grammatical concepts that they already know extremely well, and with plenty of time for back-filling.
Unsurprisingly, my English classes are a little meh. I have a clear learning goal, and a good vision of what steps we need to take to get there. I think my day-do-day practice is good instruction, because a lot of my students are doing good learning, some of them actually against their will. (That was kind of a joke.) I can clean up some of my routines, particularly the beginning of class. But there's a gap, and I keep feeling like there's something missing. I look around and wonder what I'm not teaching. I may ask Annette from the ISD if she has time to observe me teach someday soon, and give me some pointers.
Subscribe to:
Posts (Atom)
