“The Shakespeare Conspiracy – A Novel: The Story of the Greatest Literary Deception of All Time – Based Entirely on Historical Facts”

Dear Commons Community,

I have just finished reading  The Shakespeare Conspiracy – A Novel: The Story of the Greatest Literary Deception of All Time – Based Entirely on Historical Facts written by Ted Bacino.  I found it a quick enjoyable read as someone who was not familiar with the theory that Shakespeare did not write his plays but simply passed on those written by  Christopher Marlowe who faked his own death to avoid being tried for treason.  

This novel, published in 2010, provides an interesting take on two questions that have plagued historians for centuries.  One, how could Christopher Marlowe, a known spy and England’s foremost playwright, be suspiciously murdered and quickly buried in an unmarked grave – just days before he was to be tried for treason?  And two, how could William Shakespeare replace Marlowe as England’s greatest playwright virtually overnight – when Shakespeare had never written anything before and was merely an unknown actor?

The reviews on it were mixed. Here is what one reviewer commented:

“This is a very well researched and meticulously thought out book. I was in awe at just how much work Bacino has put into this, with foreword, and massive appendices…

…and he’s clearly looked up every single point that he’s writing about, from plague to theatres to politics. I have to give Bacino a standing ovation simply for the work he’s done here with a foreword and a huge appendix but..

…the trouble is — it’s not really a novel. This book is really going only to appeal to historians, because those wanting an immersive novel are going to find the style jarring–as I did.

It’s more like a docu-drama.”

I found it fun and had trouble putting it down because I wanted to see how it would play out.  Try it!

Tony

Sam Donaldson Makes Prediction For Donald Trump About His ‘Bulls**t’ Outburst!

Dear Commons Community,

Veteran ABC news anchor Sam Donaldson warned last night how President Donald Trump’s use of a the word “bulls**t”  at a political rally this week may over time end up turning “a lot of people off.”  Trump said at an event in Grand Rapids, Michigan, on Thursday that “the Democrats now have to decide if they will continue to defraud the American public with this ridiculous bullshit.”  It was in reference to special counsel Robert Mueller’s report into Russian interference in the 2016 election and possible Trump campaign collusion.

Attorney General William Barr released a summary of Mueller’s report Sunday. It said Mueller did not find collusion with the Russian government. Mueller also wrote there was insufficient evidence to determine whether Trump had obstructed justice. Democrats are calling for the release of the full report.

Speaking on CNN’s “Anderson Cooper 360°,” Donaldson told John Berman that Trump’s base would not be put off by his use of the swear word because “they make excuses for him,” like claiming it was just locker room talk and everyone does it.

But others would be offended, Donaldson claimed.

“No, we don’t all do that,” he said. “And if we do, we do it privately and just with our friends some place.”

“Have I used that word you just used? Sure I have, not on television, not in public,” he said. “This is a president who does not conform to any rules or norms of civility, any rules or norms of behavior that I think most Americans want to see.”

“He doesn’t care. I think Americans care, I think he’ll find that out,” Donaldson said. “But, I could be wrong.”

I hope Trump finds it out  sooner rather than later!

Tony

 

California Lawmakers Propose a Ban on Legacy Admissions to Colleges!

Dear Commons Community,

A package of bills proposed banning legacy admissions at California universities among other reforms in the wake of the Operation Varsity Blues scandal.  Democratic lawmakers in California proposed the legislation yesterday. The reforms proposed come just weeks after the elite college cheating scandal laid bare some of the more egregious (and allegedly criminal) tactics rich families have used to get their kids in.  As reported in the Huffington Post:

“The legislative package proposed includes six bills, with one that would ban preferential admissions to California colleges for students related to donors or alumni (also known as “legacy” admissions); another bill would require any special admissions or “admissions by exception” to get approval from three administrative staff members; another would require college admissions firms and consultants to register with the Secretary of State’s office; and one proposes a study be conducted on the need for the SAT and ACT to determine admissions.

The goal of the package of bills proposed is for no student to “gain advantage over another because of their family’s wealth or social connections,” per a release from the lawmakers.  

“It’s time to close the wealthy’s side door to college,” Assemblymember Kevin McCarty (D-Sacramento) wrote on Facebook ahead of a press conference announcing the proposed reforms.

“For every student admitted through bribery, there is an honest and talented student denied access to college,” McCarty said in the press release, noting the legislative package aimed to “protect the sanctity of the admissions process.”

Earlier this month, dozens of people were charged by the FBI in an elite college admission scheme, in which wealthy parents ― including celebrity actresses Lori Loughlin and Felicity Huffman ― allegedly paid bribes to ensure that their children were accepted to schools such as Yale, Stanford and the University of Southern California. The alleged scheme included parents paying for their kids to cheat on exams and apply to schools as student athletes, whether or not they actually had any skills in the relevant sport.

The University of California said in a statement to HuffPost that it shared “legislators’ outrage and concerns over the illegal and unethical actions” of those involved in the alleged scam.

“UC and legislators are aligned on the goal of ensuring a level playing field for every applicant, regardless of income, social status, or influence,” the University of California added, noting that its policies forbid “legacy admissions.”

The California lawmakers expected the proposals to be heard in committee after next month’s spring recess, per the release.

Assemblymember Phil Ting (D-San Francisco), who was behind the bill that would ban preferential admissions to students connected to donors or alumni, said it was an issue of “fairness and equality,” according to the release.  

“We raise our kids to believe that if they work hard, all opportunities will be open to them. But that’s just not true when it comes to college,” said Ting. “We must close the side door that enables privileged families to get their children into elite colleges, taking the place of deserving students.”

As news of the bribery admissions scandal broke, many people pointed out that higher education admissions were already rigged to favor wealthy and white students ― even before reaching the point of criminality.

Experts HuffPost spoke to earlier this month pointed to wealthy families in the U.S. buying their kids’ way into college with large donations to schools, or simply by providing extra tutors, essay coaches and interview prep professionals to give their kids a leg up in getting into schools.

Perhaps the most egregious issue of all was legacy admissions, the experts noted ― or students being more likely to get accepted to a school simply because a parent or other relative attended. 

“This scandal is just the extreme, the illegal extreme, but it’s in a continuum with legacy admissions … with all these other thumbs on the scale that wealthy kids get that are legal,” Susan Dynarski, professor of economics, education and public policy at the University of Michigan, told HuffPost earlier this month.

“If you look around a college campus and you’re thinking about who got in because of a thumb on the scale, it’s the rich white legacy kids,” she added.

It will be interesting to see if other universities especially the well-endowed private colleges follow suit.

Tony

Janet Lieberman, a Force at CUNY and for Community College Education, Dies!

Dear Commons Community,

Janet Lieberman, an educational innovator who made college education more accessible to struggling high school students and recent immigrants died on March 19 in San Francisco. She was 97.  Dr. Lieberman devoted much of her career to the City University of New York and especially to LaGuardia Community College where she served as a guiding light from its inception in the 1970s.  Here is an excerpt from her New York Times obituary.

“Dr. Lieberman was present at the creation when Community College No. 9, as it was originally known, was named for Fiorello H. La Guardia, the transformative New York mayor from 1934 to 1945.

The college, part of the City University of New York, opened in 1971 in Long Island City in a refurbished plant where White Motor Company once made auto parts and Ford Instrument once manufactured range finders for naval weapons during World War II. The building was within wafting range of a Chiclets gum factory next door.

Dr. Lieberman not only helped shape the mission of LaGuardia, a two-year college that now enrolls some 45,000 students from 150 countries; she also established collaborations with other educational institutions to attract high school students who had struggled academically, or who had to hold down jobs while taking classes, or who could not afford a four-year college.

“Janet had a strong feeling that we in education were snobs and our bar was too low, and that race and class were factors in that,” said Richard K. Lieberman (no relation), a LaGuardia history professor and director of the LaGuardia and Wagner Archives, a repository of mayoral and other municipal records that Janet Lieberman helped organize.

“She knew that if we raised the bar, the students would come up to meet it,” he said.

In 1974, at a time when city officials were alarmed by plummeting high school graduation rates, Dr. Lieberman, in cooperation with the city’s Board of Education, founded the Middle College High School at LaGuardia, with support from the Ford Foundation.

The school, which couples small class sizes with intensive counseling, combines the last three years of high school with the first two of college. Typically after five years (but sometimes four), students graduate with an associate degree and can enroll in a four-year college, often transferring credits.

“The idea broke so many rules for whom community colleges should be serving, and how they might reach young, at-risk populations,” Anne-Marie McCartan wrote in “Unexpected Influence: Women Who Helped Shape the Early Community College Movement” (2017).

In the book, Ms. McCartan quoted Dr. Lieberman as saying, “We needed to make them learners and to make them aware of the possibilities.”

In 1983, Dr. Lieberman started the International High School on the LaGuardia campus for students new to the United States with limited knowledge of English.

She also teamed up with Vassar College in Poughkeepsie, N.Y., in 1985 to create the Explore Transfer program, which exchanged teachers between the colleges and enrolled LaGuardia students in Vassar classes during the summer to acclimate them to the four-year college experience and encourage them to apply there.

“Programs that she revolutionized — guiding promising underserved high school students into college, and forging collaborations between colleges to create pathways to transfer — have become models used nationwide,” Gail O. Mellow, LaGuardia’s president, said by email.

She added, “She saw opportunity where others saw none, and understood the powerful connection between boosting a young student’s self-confidence and their success.”

Janet Elaine Rubensohn was born on Oct. 2, 1921, in Flatbush, Brooklyn, the daughter of Samuel and Ida (Schubert) Rubensohn. Her mother was a schoolteacher and her father a manufacturer of Tiny Town Togs children’s clothing.

After graduating from the Berkeley Institute of Brooklyn, a prep school, she got a bachelor’s degree in economics from Barnard College in 1943 and a master’s from the City College of New York.

While she was in college, she married Allen Chase; their marriage ended in divorce. In addition to their son Randolph, she is survived by three grandchildren and four great-grandchildren. Another son, Gary Andrew Chase, died in 2007. Her second husband, Dr. Jerrold S. Lieberman, whom she married in 1957, died in 2008.

Dr. Lieberman was a public school psychologist, taught education and reading at Hunter College (also part of the City University) and earned a doctorate in educational psychology from New York University before she was recruited by LaGuardia in 1970.

“The challenge is what appealed to me,” she recalled later. “Status and money it was not.”

One of her first assignments was to reconcile the persistently steep high school dropout rate in New York City with a new policy that guaranteed high school graduates admission to the City University system.

She served as associate dean of the faculty and retired as professor emerita in LaGuardia’s social science department in 1991 but continued as special assistant to the president until 2000.

Dr. Lieberman won the 1989 Charles A. Dana Award for Pioneering Achievement in Higher Education and the 2004 Harold W. McGraw Jr. Prize in Higher Education.

In “The Wisdom Trail: In the Footsteps of Remarkable Women” (2009), Dr. Lieberman and Julie Hungar profiled a range of accomplished women, each of whom, they wrote, “has made her mark and gained gratification by making a difference.”

In her case, Dr. Lieberman said in “Unexpected Influence,” “We made a college.”

May she rest in peace!

Tony

Puerto Rico’s Governor Ricardo Rossello Warns Trump: “If the bully gets close, I’ll punch the bully in the mouth”

See the source image

Dear Commons Community,

Puerto Rico’s  Governor Ricardo Rosselló is taking a tough stance with President Trump.  After months of being careful and respectful, Rosselló voiced his frustration with the White House in an interview yesterday with CNN.

“If the bully gets close, I’ll punch the bully in the mouth,” Rosselló said when asked about a tense meeting Wednesday between members of the Trump administration and Puerto Rican officials. “It would be a mistake to confuse courtesy with [lack of] courage.”   As reported.

The Washington meeting — which was attended by White House trade adviser Peter Navarro and members of Rosselló’s government — was requested after reports that Trump was considering halting further disaster relief to the beleaguered U.S. territory.

In a Wednesday meeting with Senate Republicans, Trump said the amount of aid Puerto Rico had so far received “is way out of proportion to what Texas and Florida and others have gotten,” according to Sen. Marco Rubio, R-Fla., who attended the meeting.

Though it has already slashed benefits, Puerto Rico faces a $600 million shortfall to administer food stamps. So far the U.S. government has spent more than $6 billion on disaster relief to Puerto Rico in the wake of Hurricane Maria, which was blamed for killing more than 3,000 people. In June, Texas received $5 billion in federal aid for housing and infrastructure repairs stemming from Hurricane Harvey, which left 103 people dead.

Rosselló, who avoided criticizing Trump in a 2018 interview with Yahoo News, lashed out at the president over his latest reported comments.

“He treats us as second-class citizens, that’s for sure,” Rosselló told CNN. “And my consideration is I just want the opportunity to explain to him why the data and information he’s getting is wrong. I don’t think getting into a kicking and screaming match with the president does any good. I don’t think anyone can beat the president in a kicking and screaming match. What I am aiming to do is make sure reason prevails, that empathy prevails, that equality prevails and that we can have a discussion.”

Trump, whose administration’s response to Maria was criticized as inadequate, has long been seen as reluctant to offer aid to Puerto Rico. In October the president again signaled his disapproval of giving aid that might be used to help alleviate the financial distress the island was experiencing even before Maria hit.

Rossello is right to be frustrated with Trump.  What has happened in Puerto Rico is beyond tragic. Punch the bully, Ricardo.  He will make a lot of noise when he falls.

Tony

Megan Condis: From Fortnite to Alt-Right!

Image result for fortnite

Dear Commons Community,

As anyone who has children or grandchildren know that the game, Fortnite, is extremely popular.  My fourteen-year old grandson Michael taught me this game over the Christmas holidays while he was visiting from Seattle.  It is very engaging, maybe addictive but I could see how it would appeal to a young teenager.   Megan Condis, an assistant professor of game studies at Texas Tech University and the author of Gaming Masculinity: Trolls, Fake Geeks, and the Gendered Battle for Online Culture, has  an op-ed in today’s New York Times, raising a concern that  “it [Fortnite] plays a special role as a vector for spreading the messages of white supremacist ideology that lead to violence.”  Her point is not the violence of the game per se but the fact that white supremacists troll these types of game sites to recruit individuals usually white males to their ideology.  She presents an important issue and one especially relevant to our times.   Below is the entire op-ed.  I have not played Fortnite or any other game since my grandson went back to Seattle but I probably will have a conversation with Michael when I see him later in May.

Tony


New York Times

From Fortnite to Alt-Right

There’s a reason video games are such fertile ground for white nationalist recruitment.

By Megan Condis

Ms. Condis is an assistant professor of game studies and a gamer.

March 27, 2019

Let’s get one thing out of the way: No, the shooter who live-streamed himself killing 50 worshipers in two mosques in Christchurch, New Zealand, this month was not being serious when he wrote that “Spyro the Dragon 3 taught me ethno-nationalism” and “Fortnite trained me to be a killer.”

Rather, with this statement, the killer was ridiculing a trope that has circulated in the media increasingly since the Columbine killings in 1999: that video games are capable of brainwashing vulnerable teenagers and turning them into violent sociopaths. Some media outlets have described this section of the manifesto in particular as “trolling” or as “bait” — and it is both of those things, certainly.

And yet as a scholar who studies video game culture, I do want to talk about gaming. Because I think it plays a special role as a vector for spreading the messages of white supremacist ideology that lead to violence. And I think it’s a conversation that we can have without taking the bait — because this is not about the content of the games themselves but about the way the culture that surrounds gaming provides particularly fertile soil for sowing the seeds of resentment that grow into hate.

Modern internet-based recruitment efforts are designed around the creation of a frictionless pipeline that slowly inoculates potential converts to hate — like putting a bunch of would-be Pepe the Frogs in a slowly boiling pot.

Rather than waiting for targets to find them, recruiters go to where targets are, staging seemingly casual conversations about issues of race and identity in spaces where lots of disaffected, vulnerable adolescent white males tend to hang out. Those who exhibit curiosity about white nationalist talking points or express frustration with the alt-right’s ideological opponents such as feminists, anti-racism activists and “social justice warriors” are then escorted through a funnel of increasingly racist rhetoric designed to normalize the presence of white supremacist ideology and paraphernalia through the use of edgy humor and memes.

Of course, video games aren’t the only places online where these conversations are taking place. Social media platforms like Facebook, Twitter and YouTube are also common culprits.

But video games in particular make for an ideal recruiting venue. Why? Because they come equipped with an easy-to-understand narrative of the unwelcome “invasion” of “our spaces” that, in the right hands, can readily be expanded beyond the world of gaming.

Surveys show that in the United States, gaming is not dominated by people of one race or gender. But the stereotype of the hard-core gamer as a geeky, adolescent, straight, white male still persists within our culture — and white nationalist recruiters are great at exploiting it.

As events like the 2014 harassment campaign #GamerGate amply demonstrated, to some members of the gaming community, the increased visibility of people of color, women and L.G.B.T.Q. people in gaming circles is seen less as an expansion and more as a hostile takeover. White supremacist recruiters have recognized this feeling of resentment bubbling up and pounced, seeking out gamers who fit the stereotype. They tell those gamers that they really do represent the rightful majority within their community and that all others are either opportunistic fakers only pretending to be into games or intruders trying to ruin everything fun and unique about gaming culture with their insidious political correctness.

Planting the seeds of this narrative is the first step toward cultivating an “us versus them” mentality. According to Christian Picciolini, a former white supremacist recruiter and a co-founder of the nonprofit organization Life After Hate, this type of rhetoric can help create a politics of entitlement and resentment organized around race. So, if a young white man can be convinced that gaming “belongs” to him and that it is on the verge of being taken away, he might be more easily persuaded to accept similarly structured arguments about, say, the dangers of allowing nonwhite immigrants to take over the country under the noses of “real” Americans.

In posts in the “Gaming” section of the explicitly white nationalist message board Stormfront, participants debate among themselves about which mainstream game releases are the most amenable to white power ideology. They exchange links to servers on free chat platforms like Discord for “whites only” and to groups dedicated to white nationalism on Steam, an online gaming store. (In the wake of scathing news coverage, Steam and Discord have made efforts to try to get rid of this content.)

People with this type of ideology have also taken to creating white supremacist games of their own, either by creating explicitly neo-Nazi-themed modifications of popular titles like Doom, Counter-Strike and Stellaris or developing their own indie titles. A few standouts in the indie category include titles like Ethnic Cleansing, which allows gamers to play as a skinhead or a Klansman while participating in a “race war,” and Muslim Massacre: The Game of Modern Religious Genocide, which encouraged players to “take control of the American hero and wipe out the Muslim race.”

So if we know gaming culture is being exploited by white supremacist recruiters, where do we go from here? It can be tempting to write off video games as toxic hotbeds of hate, too tainted for the uninitiated to engage with. But this would be exactly what extremists like the New Zealand shooter want.

Despite the enormous popularity and profitability of the video game industry, gaming culture still operates in the shadows. Most media pay almost no attention to it, even though the global market for video games is currently larger than those for movies and music combined.

This inattention signals that gaming is a special place, outside the mainstream, that could indeed, with enough outright hostility, be made to “belong” to a particular group.

But this signaling is compounded, because our unwillingness to pay attention to this influential medium means that the video game industry has next to no incentive to take responsibility for the social spaces that it fosters. Our failure to take games seriously provides the companies in the games industry an excuse not to invest the time, effort and money that would be required to moderate their communities properly.

There will always be dark corners of the internet for neo-Nazis to hide in and recruit from. There will always be those who claim that gaming isn’t for everyone. But we can insist that the companies that control gaming spaces recognize that this community comes with extremism dangers and that gaming is large enough that these companies need to behave as responsible actors. We can only help to reshape and reform these communities from within. And if we turn away, we risk abandoning one of the world’s largest entertainment and communication machines to those who would use it for evil ends.

 

Network for Public Education Report on USDOE Charter School Program Finds a Troubling Pattern!

Dear Commons Community,

The Network for Public Education (NPE)  issued a report earlier this week outlining serious problems in the way the U.S. Department of Education is administering the Charter Schools Program (CSP).  Entitled,  Asleep at the Wheel: How the Federal Charter Schools Program Recklessly Takes Taxpayers and Students for a Ride, the report found a troubling pattern of insufficient applicant review, contradictions between information provided by applicants and available public data, the gifting of funds to schools with inadequate financial and governance plans, a push-out of large grants to the states with little supervision by the department, and the waste of hundreds of millions of taxpayer dollars.  As described at the NPE website,  the report compared claims made by charter grant applicants to information on state databases and school websites, and found numerous examples of federal tax dollars being misspent due to an inattentive process that routinely accepts applicants’ claims without scrutiny.  It found that it is likely that as many as one third of all charter schools receiving CSP grants never opened, or opened and shut down. In fact, the failure rates for grant-awarded charter schools in California has reached nearly four in ten.  NPE commented that:

“American taxpayers have a right to demand that their tax dollars not be wasted. Tax dollars that flow to charter schools that never opened or quickly close should not be considered the cost of doing business. And a program with a stated commitment to spread “high-quality” schools should not be a major funding source for schools that leave families in the lurch and promote discriminatory enrollment practices that increase segregation and unequal opportunity for students with disabilities, behavioral challenges or English language learner status. We cannot afford to continue to pump hundreds of millions of dollars into a program whose stewards are clearly asleep at the wheel.”

The full executive summary is below.  

Tony

—————————————————————————————————————————-

Asleep at the Wheel: How the Federal Charter Schools Program Recklessly Takes Taxpayers and Students for a Ride

Executive Summary

In 2015, the U.S. Department of Education took an unprecedented step–it told the state of Ohio to put on hold the $71 million that it gave the state for the purpose of opening more charter schools. What is even more remarkable is that the cut-off of funds was championed by Ohio Senator Sherrod Brown, who expressed concern about the charter fraud and abuse that was happening in his state.

Brown’s mistrust was well founded. Shortly after the announcement, Innovation Ohio and the Ohio Education Association issued a joint report showing that more than one in three schools that had received federal grants from the U.S. Department of Education’s Charter Schools Program (CSP) had never opened, or opened and soon closed. The report also noted that of the remaining grant-funded charters, 63 percent, were among the lowest performing schools in the state.

Was the Ohio scandal a unique event, or was it typical? That is the question this investigative report sought to an-swer, and after two months of analysis, the answer is clear.  The Ohio scandal was far from unique. We found that it is likely that as many as one third of all charter schools receiving CSP grants never opened, or opened and shut down. In fact, the failure rates for grant-awarded charter schools in California has reached nearly four in ten.

The federal outlays we examined are not modest expenditures amounting to little more than rounding errors. In its 2015 analysis, CSP stated that, since its inception in 1994, the program had provided $3.3 billion to fund the startup, replication, and expansion of charter schools, creating 40 percent of operational public charter schools in the nation. We estimate that program funding has grown to well over $4 billion. That could bring the total of the potential waste to around $1 billion.

The waste of public dollars on closed charter schools is not the only concern. Of the grant recipients that manage to stay open, we uncovered extensive evidence that raises serious questions as to whether or not these schools are truly “high-quality,” meeting the CSP goal of providing equitable access for disadvantaged students.

Through detailed examination of CSP’s application process, and by comparing claims made by charter grant ap-plicants to information on state databases and school websites, we found numerous examples of federal tax dol-lars being misspent due to an inattentive process that routinely accepts applicants’ claims without scrutiny. In short, despite the scandal of Ohio and numerous critical reports by their own Office of Inspector General, the U.S. Department of Education has been asleep at the wheel when it comes to the management and supervision of hundreds of millions of taxpayer dollars every year.

Below is a summary of our findings:

1. Hundreds of millions of federal taxpayer dollars have been awarded to charter schools that never opened or opened and then shut down. In some cases, schools have received federal funding even before securing their charter. Our investigation barely skimmed the surface of the hundreds of charter school grant recipients that never opened or opened but then closed. Among the scores of schools examined, we found a Seattle private school that converted to a charter with grant money only to shortly flip back to a private school, leaving 90 economically dis-advantaged children scrambling to find a new school mid-year. We found two Delaware charter schools started by the same financial firm that won multiyear grants two years apart from each other. One opened its doors but closed midyear, and the second never opened at all. We found a Hawaii charter that won a CSP award in 2016 that has yet to find a location, while its website continues to say it is accepting new enrollees. Of the schools awarded grants directly from the department between 2009 and 2016, nearly one in four either never opened or shut its doors. The CSP’s own analysis from 2006-2014 of its direct and state pass-through funded programs found that near-ly one out of three awardees were not currently in operation by the end of 2015.

2. The CSP’s grant approval process appears to be based on the application alone, with no attempt to verify the information presented. Schools have been approved for grants despite serious concerns noted by review-ers. The CSP’s review process to award grants does not allow the verification of applicants’ claims, thus leading to what award-winning, New York Times journalist Michael Winerip referred to as an “invitation for fiction writing.” This process resulted in numerous examples of awardees that claim they seek to enroll high percentages of mi-nority and disadvantaged students, even while their programs and policies are designed to draw from advantaged populations. Finally, we found instances where achievement and/or demographic data on applications were cherry-picked or massaged, with reviewers instructed to accept what was written as fact.

3. Grants have been awarded to charter schools that establish barriers to enrollment, discouraging or deny-ing access to certain students.Multiple schools we examined enroll smaller percentages of students with disabilities and students who are Eng-lish language learners than the surrounding schools. Some appear to be designed to encourage “white flight” from public schools. Thirty-four California charter schools that received CSP grants appear on the ACLU of Southern California’s list of charters that discriminate—in some cases illegally—in admissions, and 20 CSP funded Arizona charters appear on a similar list created by the Arizona ACLU. One Pennsylvania charter receiving multi-ple grants totaling over one million dollars from CSP states on its website that its programs are “limited to stu-dents with mild handicaps.”

4. Recommendations by the Office of the Inspector General have been largely ignored or not sufficiently ad-dressed.We reviewed numerous OIG audits that found significant concerns over how CSP money is spent and about the general lack of monitoring the Department carries out to ensure those funds contribute to the intended goals of the grants. Each audit includes specific recommendations to correct this lack of oversight. But not only is there little evidence the department has adopted any of these recommendations; the current Secretary has denied re-sponsibility for oversight, believing that it falls outside the federal government’s purview—even though this is a federal grants program.

5. The department does not conduct sufficient oversight of grants to State Entities or State Education Agen-cies, despite repeated indications that the states are failing to monitor outcomes or offer full transparency on their sub grants. Although the vast majority of public charter school grants are awarded to state education agencies (SEAs), our investigation reveals that the Department has shown little oversight when SEAs pass that funding along directly to individual charters or charter organizations as subgrants. We found a continuing record of subgrantee schools that never opened or closed quickly, schools that blatantly discriminate in their discipline, curricular, and enrollment practices, schools that engage in outright fraud as well schools that engage in related-party transactions that result in private individuals and companies pocketing huge sums of money at taxpayer expense.

6. The CSP’s grants to charter management organizations are beset with problems including conflicts of in-terest and profiteering.The Office of the Inspector General’s 2016 audit of CSP funded CMO’s and/or their related schools found that of the 33 schools they reviewed, 22 had one or more of the following: conflicts of interest between the CMO and the charter, related-party transactions and insufficient segregation of duties. We found troubling examples of CMOs that received massive grants that engaged in practices that push-out low-performing students, violate the rights of students with disabilities and cull their student bodies through policies, programs and requests for parental donations.

7. Under the current administration, while Congressional funding for the CSP rises, the quality of the ap-plications and awardees has further declined.Based on our review of grant awards to SEAs and non-SEAs in 2017 and 2018, we contend the quality of the ap-plications and the receiving grantees are likely getting worse, and the department’s willingness to provide over-sight has nearly disappeared, which may result in increased fraud, mismanagement and charter failure.

Recommendations

Our investigation finds the U.S. Department of Education has not been a responsible steward of taxpayer dollars in its management of the CSP. Based on what we found, we believe it is likely that one billion dollars of federal “seed money” has been wasted on charters that never opened or shut their doors. We were equally dismayed to find that many of the CSP-funded charter schools that survived did not fulfill their stated mission, especially in regard to enrolling proportionate numbers of disadvantaged youth. As public dollars are pulled from public schools and a more disadvantaged student body is left behind, the students who attend their neighborhood school have fewer resources and greater challenges.

Finally we fear that the department’s indifference to accountability and its unwillingness to supervise the hun-dreds of millions of taxpayer dollars that flow through the program are likely to increase under the current Secre-tary who presses for choice for the sake of choice, regardless of the cost to the American taxpayers and the dis-ruption it causes to children and families.

Therefore, we recommend that Congress end funding for new charter grants coming from CSP. We also recom-mend thorough audits of previous grant awards, steps to ensure grant awards still under term are being respon-sibly carried out and that misspent money is returned. We cannot afford to continue to pump hundreds of millions of dollars into a program whose stewards are clearly asleep at the wheel.

Colleges Moving to Outsource More Services!

Dear Commons Community,

Goldie Blumestyk has an article in the online edition of The Chronicle of Higher Education highlighting a growing movement among college administrators to outsource services.  Here is an excerpt:

“If you hear the term [outsourcing]  and think “dining services” or “bookstore,” you’re not wrong. But you’d be overlooking a range of other services that colleges increasingly eye as ripe for partnership with outside parties. Among them: managing online programs, predictive-analytics systems, skills training and boot camps, and even career counseling.

Those findings come from a new poll by The Chronicle and P3-Edu, a coming conference hosted by George Mason University. (P3 stands for “public-private partnership”; the term is commonly used even when university partners are private too.) The survey collected responses from 249 presidents, provosts, and chief financial officers.

I’ve written before about the issues that arise when colleges align with companies for services that touch the classroom and other areas close to the academic core.

The chart above shows some top-line results from the survey. (Respondents could list more than one answer.) It’s not a scientific sampling, but it does offer a snapshot of the state of play. 

What I found even more interesting were the anonymous comments people provided about their recent experiences with P3s, which laid out the promise of the partnerships and the reservations about them.

Let’s start with those reservations. (C’mon, you knew I’d do that.) More than 80 answered that question. Losing control of operations and finding companies’ missions misaligned with the institution’s were the fears that came up the most. College leaders mentioned the potential for communication failures and damage to their reputations and brands.

Here’s a sampling:

“Always a concern that control is diluted. As a public institution, accountability is never diluted.”

“Companies are profit-making. We’re a school. We do want efficiencies, but our missions are different.”

“Basically selling our soul to the devil.”

Several respondents cited concerns about costs, forgoing too much in income, and whether partnerships would be sustainable over the life of their contracts. As one wrote: “Turnover in the C-suite can undermine a partnership.” As another put it: “Early in the process, everyone is ‘in love’ and life is good. Later in the process, sorry to say, ‘divorces’ occur.”

Those are valid concerns, and they barely touch on some of the more intricate possibilities — say, if a vendor’s plan for building online enrollment doesn’t mesh with an academic department’s belief in how a particular degree program should be structured.

Still, there’s no denying that colleges see value in these outside parties. Two-thirds of college leaders said the “unique competencies” of private parties were a top reason they would look to them — or had looked to them already — as partners.

The rising interest in P3s is also why George Mason decided to hold its P3 conference again this spring.

About half of the institutions that responded to the survey enroll fewer than 2,000 students; the rest are larger. Nearly three-quarters reported endowments no greater than $250 million; 14 percent were over $1 billion.

A few disparities stood out. Smaller institutions were more likely to cite “superior service to in-house alternatives” as a reason to pursue partnerships. And while 46 percent of smaller institutions cited “online program expansion” as an area of interest, only 36 percent of the larger ones did.”

Good information for those administrators considering outsourced services.

Tony

 

Education Secretary Betsy DeVos Proposes to Slash Special Olympics Program in Favor of Charter Schools Funding!

Image result for betsy devos

Dear Commons Community,

During a Congressional hearing yesterday, Democrats grilled Betsy DeVos over her latest budget proposal, which boosts appropriations for charter schools while slashing $17.6 million in funding for the Special Olympics.  During a contentious hearing before the House education appropriations subcommittee, Rep. Mark Pocan, D-Wis., pushed DeVos on the proposed cuts to the Special Olympics, a sports competition for disabled athletes.  As reported by Yahoo News:

“Do you know how many kids are going to be affected by that cut, Madam Secretary?” Pocan asked.

“Mr. Pocan, let me just say again, we had to make some difficult decisions with this budget —” DeVos replied.

“Again, this is a question about many kids, not about the budget —” Pocan interjected.

“I don’t know the number —” said DeVos.

“It’s 272,000 kids,” Pocan interjected.

“Let me just say that I think Special Olympics is an awesome organization, one that is well supported by the philanthropic sector as well,” DeVos said.

Overall, DeVos’s proposal cuts the education budget by $7 billion, or 10 percent of the current level of funding. At the same time, it would raise charter school funding by $60 million.

As word of the exchange and DeVos’s budget request spread across social networking sites, others began to criticize the plan.

Asked to comment on DeVos’s budget proposal, Special Olympics communications director Tara Baker told Yahoo News that progress has been made in recent years in “eliminating the stigma, stereotypes, isolation and discrimination” for people with intellectual disabilities.

“We ask federal, state and local governments to join Special Olympics in remaining vigilant against any erosion of provisions that have made a substantial difference in the lives of people with ID [intellectual disabilities],” Baker said in an emailed statement. “As is the case each year after the President presents his budget to Congress, we engage in opportunities, such as our annual Capitol Hill Day activities, to educate lawmakers about why grant funding for our health and education programming is critical to protecting and increasing access to these services for people with intellectual disabilities. We look forward to continuing to raise awareness among U.S. government officials about the important work that Special Olympics is doing in the United States and around the world.”

The budget submitted by the administration and Cabinet agencies invariably gets overhauled by Congress, and is usually regarded as symbolic. Still this was another embarrassing performance by the education secretary.

Tony

Dark Side of Artificial Intelligence in Health Care!

Dear Commons Community,

In an article yesterday, the New York Times commented on a growing concern in the use of artificial intelligence (A.I.) applications in diagnosing health issues. Entitled, Warnings of a Dark Side to A.I. in Health Care, the article raises an alarm that scientists are beginning to worry that with just tiny tweaks to data, neural networks can be fooled into committing “adversarial attacks” that mislead rather than help.  As described in the article:

A new breed of artificial intelligence technology is rapidly spreading across the medical field, as scientists develop systems that can identify signs of illness and disease in a wide variety of images, from X-rays of the lungs to C.A.T. scans of the brain. These systems promise to help doctors evaluate patients more efficiently, and less expensively, than in the past.

Similar forms of artificial intelligence are likely to move beyond hospitals into the computer systems used by health care regulators, billing companies and insurance providers. Just as A.I. will help doctors check your eyes, lungs and other organs, it will help insurance providers determine reimbursement payments and policy fees.

Ideally, such systems would improve the efficiency of the health care system. But they may carry unintended consequences, a group of researchers at Harvard and M.I.T. warns.

In a paper published on Thursday in the journal Science, (see below for full article) the researchers raise the prospect of “adversarial attacks” — manipulations that can change the behavior of A.I. systems using tiny pieces of digital data. By changing a few pixels on a lung scan, for instance, someone could fool an A.I. system into seeing an illness that is not really there, or not seeing one that is.

Software developers and regulators must consider such scenarios, as they build and evaluate A.I. technologies in the years to come, the authors argue. The concern is less that hackers might cause patients to be misdiagnosed, although that potential exists. More likely is that doctors, hospitals and other organizations could manipulate the A.I. in billing or insurance software in an effort to maximize the money coming their way.”

We are entering new territory in the use of A.I. in all kinds of applications most of which we hope will be beneficial.  This article is an early alert to some of the “dark” issues that A.I. will bring.

Tony

———————————————————————————————————-

Science

Adversarial attacks on medical machine learning

In Section: Policy Forum | Machine Learning

Emerging vulnerabilities demand new conversations

By Samuel G. Finlayson 1, John D. Bowers 2, Joichi Ito 3, Jonathan L. Zittrain 2, Andrew L. Beam 4, Isaac S. Kohane 1

With public and academic attention increasingly focused on the new role of machine learning in the health information economy, an unusual and no-longer-esoteric category of vulnerabilities in machinelearning systems could prove important. These vulnerabilities allow a small, carefully designed change in how inputs are presented to a system to completely alter its output, causing it to confidently arrive at manifestly wrong conclusions. These advanced techniques to subvert otherwise-reliable machine-learning systems—so-called adversarial attacks—have, to date, been of interest primarily to computer science researchers ( Display footnote number: 1 ). However, the landscape of often-competing interests within health care, and billions of dollars at stake in systems’ outputs, implies considerable problems. We outline motivations that various players in the health care system may have to use adversarial attacks and begin a discussion of what to do about them. Far from discouraging continued innovation with medical machine learning, we call for active engagement of medical, technical, legal, and ethical experts in pursuit of efficient, broadly available, and effective health care that machine learning will enable.

In medical diagnostics and decision support, machine-learning systems appear to have achieved diagnostic parity with physicians on tasks in radiology, pathology, dermatology, and ophthalmology ( Display footnote number: 2 ). In 2018, the U.S. Food and Drug Administration (FDA) approved marketing for the first-ever autonomous artificial intelligence (AI) diagnostic system and indicated that they are “actively developing a new regulatory framework to promote innovation in this space” ( Display footnote number: 3 ). Regulators have articulated plans for integrating machine learning into regulatory decisions by way of computational surrogate end points and so-called “in silico clinical trials.”

Under the United States’ health care model, some of the most direct impacts of machinelearning algorithms come in the context of insurance claims approvals. Billions of medical claims are processed each year, with approvals and denials directing trillions of dollars and influencing treatment decisions for millions of patients. In addition to dictating the availability of patient care, claims approval is vested with competing financial interests, with providers seeking to maximize and payers seeking to minimize reimbursement ( Display footnote number: 4 ). Given the volume and value of processing medical claims, it is unsurprising that many providers engage in creative and often fraudulent practices to increase their revenue ( Display footnote number: 5 ). For their part, insurance companies and their contractors have invested in extensive machine-learning infrastructure for billing code processing. Although much of our discussion highlights financial incentives specific to the fee-for-service model in the United States, the implications of algorithmic vulnerabilities have broad relevance.

DEEP VULNERABILITIES

Adversarial examples are inputs to a machine-learning model that are intentionally crafted to force the model to make a mistake. Adversarial inputs were first formally described in 2004, when researchers studied the techniques used by spammers to circumvent spam filters ( Display footnote number: 6 ). Typically, adversarial examples are engineered by taking real data, such as a spam advertising message, and making intentional changes to that data designed to fool the algorithm that will process it. In the case of text data like spam, such alterations may take the form of adding innocent text or substituting synonyms for words that are common in malignant messages. In other cases, adversarial manipulations can come in the form of imperceptibly small perturbations to input data, such as making a humaninvisible change to every pixel in an image. Researchers have demonstrated the existence of adversarial examples for essentially every type of machine-learning model ever studied and across a wide range of data types, including images, audio, text, and other inputs ( Display footnote number: 1 ).

Cutting-edge adversarial techniques generally use optimization theory to find small data manipulations likely to fool a targeted model. As a proof of concept in the medical domain, we recently executed successful adversarial attacks against three highly accurate medical image classifiers ( Display footnote number: 7 ). The top figure provides a real example from one of these attacks, which could be fairly easily commoditized using modern software. On the left, an image of a benign mole is shown, which is correctly flagged as benign with a confidence of >99%. In the center, we show what appears to be random noise, but is in fact a carefully calculated perturbation: This “adversarial noise” was iteratively optimized to have maximum disruptive effect on the model’s interpretation of the image without changing any individual pixel by more than a tiny amount. On the right, we see that despite the fact the perturbation is so small as to be visually imperceptible to human beings, it fools the model into classifying the mole as malignant with 100% confidence. It is important to emphasize that the adversarial noise added to the image is not random and has near-zero probability of occurring by chance. Thus, such adversarial examples reflect not that machine-learning models are inaccurate or unreliable per se but rather that even otherwise-effective models are susceptible to manipulation by inputs explicitly designed to fool them.

Adversarial attacks constitute one of many possible failure modes for medical machine-learning systems, all of which represent essential considerations for the developers and users of models alike. From the perspective of policy, however, adversarial attacks represent an intriguing new challenge, because they afford users of an algorithm the ability to influence its behavior in subtle, impactful, and sometimes ethically ambiguous ways.

Deliberately crafting a noise-based adversarial example targeting a visual diagnostic algorithm, as in the top figure, would amount to overt fraud. However, the bottom figure demonstrates that adversarial techniques include a broad range of perturbations and can be applied across a vast number of input mediums. Some of these perturbations seem to be far less explicitly manipulative than the attack depicted in the top figure. As the bottom figure shows, minimal, but precise, adjustments such as rotating images to a specific angle have been shown to amount to effective adversarial attacks even against modern con-volutional neural networks ( Display footnote number: 8 ). In natural-language processing, substitution of carefully selected synonyms can be sufficient to fool algorithms such as the hypothetical opioid risk algorithm (see the bottom figure) ( Display footnote number: 9 ). In the case of structured data such as billing codes, adversarial techniques could be used to automate the discovery of code combinations that maximize reimbursement or minimize the probability of claims rejection.

Because adversarial attacks have been demonstrated for virtually every class of machine-learning algorithms ever studied, from simple and readily interpretable methods such as logistic regression to more complicated methods such as deep neural networks ( Display footnote number: 1 ), this is not a problem specific to medicine, and every domain of machine-learning application will need to contend with it. Researchers have sought to develop algorithms that are resilient to adversarial attacks, such as by training algorithms with exposure to adversarial examples or using clever data processing to mitigate potential tampering ( Display footnote number: 1 ). Early efforts in this area are promising, and we hope that the pursuit of fully robust machine-learning models will catalyze the development of algorithms that learn to make decisions for consistently explainable and appropriate reasons. Nevertheless, current general-use defensive techniques come at a material degeneration of accuracy, even if sometimes at improved explainability ( Display footnote number: 10 ). Thus, the models that are both highly accurate and robust to adversarial examples remain an open problem in computer science.

These challenges are compounded in the medical context. Medical information technology (IT) systems are notoriously difficult to update, so any new defenses could be difficult to roll out. In addition, the ground truth in medical diagnoses is often ambiguous, meaning that for many cases no individual human can definitively assign the true label between, say, “benign” and “cancerous” on a photograph of a mole. This could enable bad actors to selectively perturb borderline cases without easy means of review, consistently nudging scales in their direction.

EXISTING ADVERSARIAL BEHAVIOR

Cutting-edge adversarial attacks have yet to be found in the health care context, though less formalized adversarial practice is extremely common. This existing activity suggests that incentives for more sophisticated adversarial attacks may already be in place. To illustrate existing behaviors, we look to the modern U.S. medical billing industry.

Medical claims codes determine reimbursement for a patient visit after they have been approved by a payer. To evaluate these claims, payers typically leverage automated fraud detectors, powered increasingly by machine learning. Health care providers have long exerted influence on payers’ decisions (the algorithmic outputs) by shaping their records (and accompanying codes) of patient visits (the inputs) ( Display footnote number: 5 ).

CREDITS: (GRAPHIC) N. CARY/SCIENCE; (PHOTOS) ISIC/CC-0

At the extreme of this tactical shaping of a patient presentation is medical fraud, a $250 billion industry ( Display footnote number: 11 ). Although some providers may submit overtly fictitious medical claims, misrepresentation of patient data often takes much more subtle forms. For example, intentional upcoding is the practice of systematically submitting billing codes for services related to, but more expensive than, those that were actually performed. This practice is rampant and is just one of many questionable billing practices deployed in clinical practice. Some physicians, for example, are inclined to report exaggerated anesthesia times to increase revenue ( Display footnote number: 12 ).

In other circumstances, subtle billing code adjustments fall within a gray zone between fraud and well-intentioned best practices. In one striking example, the website of the Endocrine Society recommends that providers do not bill for the International Classification of Diseases (ICD) code 277.77 (metabolic syndrome) in patients with obesity, as this combination of code and condition is likely to result in a denial of coverage ( Display footnote number: 13 ). Instead, the Society recommends billing for codes corresponding to specific diseases that make up metabolic syndrome, such as hypertension. In other words, providers are not encouraged to add fraudulent claims but are encouraged to avoid adding a true claim that an insurance company would be likely to reject in combination with another. This recommendation is arguably motivated to serve the patients seeking coverage, not only the doctors receiving reimbursement. However, it highlights both a moral gray zone and the type of strategy that providers might use to achieve the same end result as upcoding without ever committing overt fraud.

A GROWTH INDUSTRY

As the machine-learning tool kit used by insurance companies and their contractors continues to expand, the same dynamics that favor creative billing practices in the present may expand to include more sophisticated adversarial attacks. Adversarial methods could allow billing teams to scale up upcoding practices without getting flagged by fraud detectors. Many insurance companies are beginning to require other data types such as imaging and text to prove that claims are valid. As they do so, other styles of adversarial attacks may be used as well to try to continue to dodge detection.

For example, if an insurance company requires that an image from a mole be run through a melanoma classifier before approving reimbursement for an excision, fraudsters may at first be inclined to submit moles from different patients to achieve approval. If insurance companies then begin utilizing human audits or technical tests to try to ensure that the images are coming from the correct patient, the next round would be to move to full adversarial attacks with imperceptible alterations, such as in the top figure. Simpler techniques such as the rotation in the bottom figure could constitute an ethical gray zone—given that a dermatologist could, in theory, hold the camera at any angle.

“An essential question remains: when and how to intervene.”

Potential applications of adversarial attacks in the medical context go far beyond insurance fraud, encompassing a spectrum of motivations. For instance, many adversarial attacks could be motivated by a desire to provide high-quality care. A hypothetical illustration can be drawn from the opioid crisis. In response to rampant overprescription of opiates, insurance companies have begun using predictive models to deny opiate prescription filings on the basis of risk scores computed at the patient or provider level. What if a physician, certain that she had a patient who desperately needed oxycontin but would nonetheless run afoul of the prescription authorization algorithm, could type a special pattern of algorithmically selected billing codes or specific phrases into the record to guarantee approval?

Companies might face temptations in the context of drug and device approvals. Regulatory bodies, including the FDA, have expressed interest in using algorithmic biomarkers as end points in clinical trials and other approval processes. If this is realized, adversarial examples could provide a means for companies to bias trial outcomes in their favor. For example, if a regulator requires matched images or wearable readouts from each patient before and after treatment, trialists could inject adversarial noise into posttreatment data, securing the algorithmically measured results they desired. Motivations could be complex—whereas some trialists would be motivated by the potential for a big payday, others might turn to adversarial attacks to “adjust” borderline trial results for products that might save lives.

A PATH FORWARD

An essential question remains: when and how to intervene. Here, the early history of the internet offers a lesson. The approach to network architecture introduced at the advent of the internet was centered around the deferral of problems. In their essential 1984 paper, Saltzer et al. describe a design ethos whereby problems are to be solved at the end points of a network by users rather than preemptively within the architecture of the network itself ( Display footnote number: 14 ). There is frequently an advantage (in terms of simplicity, flexibility, and scalability) to leaving future problems unsolved until their time has come. Another description for this is the “procrastination principle” ( Display footnote number: 15 ).

The procrastination principle frames a difficult question: Should the adversarial-examples problem in health care systems be addressed now—in the early, uncertain days of medical AI algorithms—or later, when algorithms and the protocols governing their use have been firmed up? At best, acting now could equip us with more resilient systems and procedures, heading the problem off at the pass. At worst, it could lock us into inaccurate threat models and unwieldy regulatory structures, stalling developmental progress and robbing systems of the flexibility they need to confront unforeseen threats.

One regulatory response might be to insist on forestalling implementation of vulnerable algorithms until they are made adequately resilient. However, given the potential of these algorithms to improve health care delivery for millions, this strategy might do more harm than good, and adequate resiliency is not imminent. Generally resilient algorithms confront an unfortunate reality familiar to cybersecurity practitioners: Breaking systems is often easier than protecting them. This is because defenses must secure against all conceivable present and future attacks, whereas attacks need only defeat one or more specific defenses. Like hack-proofing, defending against adversarial examples is a cat-and-mouse game.

Nevertheless, there are incremental defensive steps that might be taken in the short term given sufficient political and institutional will. Best practices in hospital labs are already enforced through regulatory measures such as Clinical Laboratory Improvement Amendments, which could easily be amended or extended to cover best practices engineered against adversarial attacks. For example, in situations in which tampering with clinical data or images might be possible, a “fingerprint” hash of the data might be extracted and stored at the moment of capture. Comparison of this original hash to that of the data fed through a targeted algorithm would allow investigators to determine if that data had been tampered with or changed after acquisition. Such an intervention would rely on a health IT infrastructure capable of supporting the capture and secure storage of these hashes. But as a strictly regulated field with a focus on accountability and standards of procedure, health care may be very well suited to such adaptations.

The coalescence of strong motives to manipulate algorithms and the rapid proliferation of algorithms vulnerable to manipulation makes health care a plausible ground zero for the emergence of adversarial examples into real-world practice. As adversarial examples emerge across a range of domains, we will have to make choices again and again about whether and how to intervene early at the risk of stifling development, and how to balance the promises of ubiquitous machine learning against liabilities imposed by these emerging vulnerabilities. And the stakes will remain high—autonomous vehicles and AI-driven weapons systems will be just as susceptible. A clear-eyed and principled approach to adversarial attacks in the health care context—one which builds the groundwork for resilience without crippling rollout and sets ethical and legal standards for line-crossing behavior—could serve as a critical model for these future efforts.

1Harvard Medical School, Boston, MA 02115, USA.

2Harvard Law School, Cambridge, MA 02138, USA.

3Massachusetts Institute of Technology Media Lab, Cambridge, MA 02139, USA.

4Harvard T.H. Chan School of Public Health, Boston, MA 02115, USA. Email: samuel_finlayson@hms.harvard.edu; isaac_kohane@hms.harvard.edu

REFERENCES AND NOTES

  1. B. Biggio, F. Roli, Pattern Recognit. 84, 317 (2018).
  2. T. Ching et al., J. R. Soc. Interface 15, 20170387 (2018).
  3. S. Gottlieb, “FDA’s comprehensive effort to advance new innovations: Initiatives to modernize for innovation,” FDA Voices, 29 August 2018; www.fda.gov/NewsEvents/Newsroom/FDAVoices/ucm619119.htm.
  4. A. S. Kesselheim, T. A. Brennan, N. Engl. J. Med. 352, 855 (2005).
  5. M. K. Wynia, D. S. Cummins, J. B. VanGeest, I. B. Wilson, JAMA 283, 1858 (2000).
  6. N. Dalvi et al., in KDD ’04: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, New York, 2004).
  7. S. G. Finlayson et al., arXiv:1804.05296 [cs.CR] (15 April 2018).
  8. L. Engstrom, B. Tran, D. Tsipras, L. Schmidt, A. Madry, arXiv:1712.02779 [cs.LG] (7 December 2017).
  9. J. Li, S. Ji, T. Du, B. Li, T. Wang, arXiv:1812.05271 [cs.CR] (13 December 2018).
  10. D. Tsipras, S. Santurkar, L. Engstrom, A. Turner, A. Madry, arXiv:1805.12152 [stat.ML] (20 May 2018).
  11. A. Jain, S. Nundy, K. Abbasi, BMJ 348, g4181 (2014).
  12. E. C. Sun, R. P. Dutton, A. B. Jena, JAMA Netw. Open 1, e184288 (2018).
  13. K. Reynolds, P. Muntner, V. Fonseca, Diabetes Care 28, 1831 (2005).
  14. J. H. Saltzer, D. P. Reed, D. D. Clark, ACM Trans. Comput. Syst. 2, 277 (1984).
  15. J. Zittrain, The Future of the Internet and How to Stop It (Yale Univ. Press, 2008).

ACKNOWLEDGMENTS

S.G.F. was supported by training grant T32GM007753 from the National Institute of General Medical Science. A.L.B. and I.S.K. contributed equally to this work. The content is solely the responsibility of the authors and does not necessa rily represent the official views of the National Institute of General Medical Sciences or the National Institutes of Health.

10.1126/science.aaw4399