The relative decline of American education at the elementary- and high-school levels has long been a national embarrassment as well as a threat to the nation's future. Once upon a time, American students tested better than any other students in the world. Now, ranked against European schoolchildren, America does about as well as Lithuania, behind at least 10 other nations. Within the United States, the achievement gap between white students and poor and minority students stubbornly persists—and as the population of disadvantaged students grows, overall scores continue to sag.

For much of this time—roughly the last half century—professional educators believed that if they could only find the right pedagogy, the right method of instruction, all would be well. They tried New Math, open classrooms, Whole Language—but nothing seemed to achieve significant or lasting improvements.

Yet in recent years researchers have discovered something that may seem obvious, but for many reasons was overlooked or denied. What really makes a difference, what matters more than the class size or the textbook, the teaching method or the technology, or even the curriculum, is the quality of the teacher. Much of the ability to teach is innate—an ability to inspire young minds as well as control unruly classrooms that some people instinctively possess (and some people definitely do not). Teaching can be taught, to some degree, but not the way many graduate schools of education do it, with a lot of insipid or marginally relevant theorizing and pedagogy. In any case the research shows that within about five years, you can generally tell who is a good teacher and who is not.

It is also true and unfortunate that often the weakest teachers are relegated to teaching the neediest students, poor minority kids in inner-city schools. For these children, teachers can be make or break. "The research shows that kids who have two, three, four strong teachers in a row will eventually excel, no matter what their background, while kids who have even two weak teachers in a row will never recover," says Kati Haycock of the Education Trust and coauthor of the 2006 study "Teaching Inequality: How Poor and Minority Students Are Shortchanged on Teacher Quality."

Nothing, then, is more important than hiring good teachers and firing bad ones. But here is the rub. Although many teachers are caring and selfless, teaching in public schools has not always attracted the best and the brightest. There once was a time when teaching (along with nursing) was one of the few jobs not denied to women and minorities. But with social progress, many talented women and minorities chose other and more highly compensated fields. One recent review of the evidence by McKinsey & Co., the management consulting firm, showed that most schoolteachers are recruited from the bottom third of college-bound high-school students. (Finland takes the top 10 percent.)

At the same time, the teachers' unions have become more and more powerful. In most states, after two or three years, teachers are given lifetime tenure. It is almost impossible to fire them. In New York City in 2008, three out of 30,000 tenured teachers were dismissed for cause. The statistics are just as eye-popping in other cities. The percentage of teachers dismissed for poor performance in Chicago between 2005 and 2008 (the most recent figures available) was 0.1 percent. In Akron, Ohio, zero percent. In Toledo, 0.01 percent. In Denver, zero percent. In no other socially significant profession are the workers so insulated from accountability. The responsibility does not just fall on the unions. Many principals don't even try to weed out the poor performers (or they transfer them to other schools in what's been dubbed the "dance of the lemons"). Year after year, about 99 percent of all teachers in the United States are rated "satisfactory" by their school systems; firing a teacher invites a costly court battle with the local union.

Over time, inner-city schools, in particular, succumbed to a defeatist mindset. The problem is not the teachers, went the thinking—it's the parents (or absence of parents); it's society with all its distractions and pathologies; it's the kids themselves. Not much can be done, really, except to keep the assembly line moving through "social promotion," regardless of academic performance, and hope the students graduate (only about 60 percent of blacks and Hispanics finish high school). Or so went the conventional wisdom in school superintendents' offices from Newark to L.A. By 1992, "there was such a dramatic achievement gap in the United States, far larger than in other countries, between socioeconomic classes and races," says Kate Walsh, president of the National Council on Teacher Quality. "It was a scandal of monumental proportions, that there were two distinct school systems in the U.S., one for the middle class and one for the poor."

In the past two decades, some schools have sprung up that defy and refute what former president George W. Bush memorably called "the soft bigotry of low expectations." Generally operating outside of school bureaucracies as charter schools, programs like KIPP (Knowledge Is Power Program) have produced inner-city schools with high graduation rates (85 percent). KIPP schools don't cherry-pick—they take anyone who will sign a contract to play by the rules, which require some parental involvement. And they are not one-shot wonders. There are now 82 KIPP schools in 19 states and the District of Columbia, and, routinely, they far outperform the local public schools. KIPP schools are mercifully free of red tape and bureaucratic rules (their motto is "Work hard. Be nice," which about sums up the classroom requirements). KIPP schools require longer school days and a longer school year, but their greatest advantage is better teaching.

It takes a certain kind of teacher to succeed at a KIPP school or at other successful charter programs, like YES Prep. KIPP teachers carry cell phones so students can call them at any time. The dedication required makes for high burnout rates. It may be that teaching in an inner-city school is a little like going into the Special Forces in the military, a calling for only the chosen few.

Yet those few are multiplying. About 20 years ago, a Princeton senior named Wendy Kopp wrote her senior thesis proposing an organization to draw graduates from elite schools into teaching poor kids. Her idea was to hire them for just a couple of years, and then let them move on to Wall Street or wherever. Today, Teach for America sends about 4,100 grads, many from Ivy League colleges, into inner-city schools every year. Some (about 8 percent) can't hack it, but most (about 61 percent) stay in teaching after their demanding two-year tours. Two thirds of TFA's 17,000 alumni are still involved in education and have become the core of a reform movement that is having real impact. The founders of KIPP, Mike Feinberg and Dave Levin, are TFA products. So is the most aggressive reformer in education today, Michelle Rhee, the education chancellor of the District of Columbia, who is trying to loosen the hold of the teachers' union on a school system that for years had the highest costs and worst results in the nation. (See following story.)

It is difficult to dislodge the educational establishment. In New Orleans, a hurricane was required: since Katrina, New Orleans has made more educational progress than any other city, largely because the public-school system was wiped out. Using nonunion charter schools, New Orleans has been able to measure teacher performance in ways that the teachers' unions have long and bitterly resisted. Under a new Louisiana law, New Orleans can track which ed schools produce the best teachers, forcing long-needed changes in ed-school curricula. (The school system of Detroit is just as broken as New Orleans's was before the storm—but stuck with largely the same administrators, the same unions, and the same number of kids, and it has been unable to make any progress.)

The teachers' unions—the National Education Association (3.2 million members) and the American Federation of Teachers (1.4 million members)—are major players in the Democratic Party at the national and local levels. So it is extremely significant—a sign of the changing times—that the Obama administration has taken them on. Education Secretary Arne Duncan is dangling money as an incentive for state legislatures to weaken the grip of the teachers' unions. To compete for $4.3 billion in federal aid under the Race to the Top program, states get extra points for getting rid of caps on the number of charter schools (a union favorite, since charter schools are often nonunion) and allowing student scores to be used in teacher evaluations. Measuring teacher performance based in part on the test scores of their pupils would seem to be a no-brainer. New Orleans uses student scores to measure teacher effectiveness. But it's prohibited by law for tenure decisions in states like New York, where the teachers' union has long been powerful.

It will take a quiet revolution to improve American schools. Some educational experts have noticed an uptick in the academic quality of new teachers, at least at the high-school level, possibly because the recession has limited other job opportunities. One of the unions, the AFT under Randi Weingarten, seems to realize that sheer obstructionism won't work. "One of the most hopeful things I've seen is that the union people don't want to spend so much time defending the not-so-good teachers anymore. I think the pressure of accountability is paying off," says Haycock of the Education Trust. "They know they will be held responsible if they are defending teachers who aren't any good."

Some teachers resent the reform movement as a bunch of elitists denigrating loyal and hardworking teachers—of whom, of course, there are many. But others welcome a boost in status that would come with higher standards. "You know, the Marine Corps never has any problem meeting its enlistment goals, because it's an elite corps, and people want to be part of something that is seen as the best," says Daniel Weisberg, general counsel of The New Teacher Project and coauthor of "The Widget Effect," a critique of teacher-evaluation programs. In Europe, where teachers enjoy more social prestige and higher salaries, schools have no trouble attracting new teachers with strong academic records.

Before the American public-education system can regain its lost crown as the envy of the world, local politicians and school administrators will have to step up. At Central Falls High School in Rhode Island, half the students drop out of school, and proficiency in math measured by state exams stands at a pitiful 7 percent among 11th graders. Under state pressure, the local superintendent, Frances Gallo, tried to improve scores by requiring teachers to work 25 minutes longer each a day, eat lunch with students once a week, and agree to be evaluated by a third party. The teachers, who make about $75,000 a year, far more than average in this depressed town, balked. They wanted another $90 an hour. So Gallo took a brave and astonishing step: she recommended firing all 74 teachers. Her boldness was praised by Education Secretary Duncan—and supported by President Obama. The teachers' union initially squawked that everyone was unfairly "blaming the teachers," but then last week backed off under a storm of media pressure and accepted the new rules requiring teachers to spend more time with the students.

The Central Falls High story was a notable breakthrough, but there is a long way to go. The media are beginning to root out the more outrageous examples: last year the Los Angeles Times ran a long series documenting the unwillingness of the education bureaucracy to fire bad teachers (like the one who told a student who attempted suicide to "carve deeper next time" and another who kept a stash of pornography and cocaine at school; both are still teaching). The Indianapolis Star reported how Lawrence Township schools had quietly laid off—with generous cash settlements and secrecy agreements—a teacher accused of sexually assaulting a student; another accused of touching students and taking photos of female students; another accused of kissing a high-school student; and a fourth with a 20-year history of complaints about injuring and harassing students, including a 1992 rape allegation. At the time the story ran last summer, all four teachers still held active teaching licenses. While these horror stories are sensational, what's also disturbing is the immunity enjoyed by the thousands of teachers who let down their students in more ordinary ways. Many more teachers are overworked, underpaid, and underappreciated. Maybe they'd get more respect if the truly bad teachers were let go.