Dear Commons Community,

The NY Times has an extensive article today on the way instructional software development companies inflate and overhype their products.  Using Carnegie Learning, a software development company started by faculty at Carnegie Mellon but now owned by the Apollo Group, the same company that owns the University of Phoenix, the article reviews claims made at the company’s website about its most popular product, Cognitive Tutor, and compares same to results reported by independent evaluations.  Looking at the company’s website, one would assume that Cognitive Tutor is the real deal and could make a difference in student achievement. The reality is that no such evaluation and testing has been done or is in fact conclusive.  The evaluations that have been done generally show no significant difference in student achievement as measured by student test scores.

Another  part of the article  reviews a federal study of field trials of instructional software  in 132 schools in 2009.

“Karen Billings, a vice president of the Software and Information Industry Association — a trade group representing many education companies — said the problem was not that companies overpromise, but that schools often do not properly deploy the products or train teachers to use them. Ms. Billings’s group helped design the field trials, in 132 schools, for the landmark 2009 government study of 10 software products, which was ordered by Congress and cost $15 million.

Then came the deflating results [essentially no significant difference]. The industry “became very hostile,” recalled Grover  Whitehurst, director of education policy at the Brookings Institution. “It seems to me,” he added, “ ‘hypocrisy’ is the right word for loving something until the results are not what you expect.”

Lastly, there is reference to: “Karen Cator, a former Apple executive who directs the Office of Educational Technology at the US Department of Education, said the clearinghouse reports on software should be “taken with a grain of salt” because they rely on standardized test scores. Those tests, Ms. Cator said, cannot gauge some skills that technology teaches, like collaboration, multimedia and research. “

I agree with her but it is the US Department of Education, her employer, that has put unbridled emphasis on standardized testing for the past ten years starting with No Child Left Behind and continuing with Race to the Top programs.   I would also question why she as a “former Apple executive” who has also served  on the board of the Software and Information Industry Association, holds the position she does.  It is a prime example of the educational industrial complex that exists in this country where  representatives of the for-profit providers of educational products and services end up in positions of great influence in shaping education policy.

Tony