Coherenta startup that creates large language models to compete with those of OpenAI, today announced the launch of a non-profit research lab: Cohere For AI. Led by Google alumn Sara Hooker, Cohere says Cohere For AI will work to solve some of the industry’s toughest challenges by contributing “fundamental research” to the open source community.
†We are very excited to lead a new non-profit AI research lab as we continue to broaden how and where research is done. There is so much to discover and our focus will be on open collaboration and contributing to fundamental research,” Hooker told TechCrunch via email. “At the same time, an important part of our work will be to expand the community and help educate the next generation of talent, by creating new entry points to work on core research. †
There has long been a concern within the AI community that not enough money is being set aside for AI research outside rich companies. A study found that ties to companies — whether funding or affiliation — in AI research grew significantly from 2008 to 2019. study showed that Google parent company Alphabet, as well as Amazon and Microsoft, hired as many as 52 tenure-track AI professors between 2004 and 2018, removing these future teachers from academic and nonprofit work.
Concentrating power within corporations has some obvious drawbacks, but one of the most disturbing is that it tends to emphasize too little values such as benevolence, justice and inclusion on the research side. A number of experts, speaking to Wired for a 2020 piece, point out that corporate AI projects have led to an “unscientific fixation” on projects that are only possible for those with access to high-performance data centers. Regardless of the domain, work within companies is often closely guarded and takes years to see the light of day – if ever it does.
†Our agenda is focused on advancing advances in machine learning questions alongside community-based research,” Hooker said.We also want to have a proactive research agenda so we can identify big challenges before they become problems that we need to fix retroactively. We are focused on a variety of different disciplines to work, for example, on reducing bias, and a very important part of the research is AI security and the robust use of models.”
Another core component that Cohere For AI hopes to build is access to computer resources, Hooker said, and in particular it helps researchers better use “cutting-edge” models to help develop their work.” as evidenced by the trends in language models (ie AI systems that understand and generate text). Just a few years ago, creating a highly advanced language model required huge computational resources. But now, thanks to academic breakthroughs and the work of the open source community, the barriers to entry are much lower than they used to be.
Road to non-profit
Backed by AI celebrities, including Pieter Abbeel, co-director of UC Berkeley’s AI lab, Cohere was founded in 2019 by a family tree team that included Aidan Gomez, Ivan Zhang and Nick Frosst. Gomez co-authored the academic paper “Attention is all you need”, who introduced the world to a fundamental AI model architecture, the Transformer. (Including high-profile systems, OpenAIs) GPT-3 and Codex are based on the Transformer architecture.) In addition to Gomez, Zhang contributed to For.ai, an open AI research collective of data scientists and engineers.
“For.ai is designed to help novice enthusiasts better connect with more experienced researchers,” Hooker said. “Many of the founding members went on to do PhDs or work in academic or industrial labs. At the time, For.ai was one of the first community-driven research groups to support independent researchers around the world. Now the Cohere team and its supporters are excited to reintroduce the original concept, but with more resources built from Cohere. †
According to Hooker, Cohere For AI will provide ways for data scientists to “meet and collaborate” through mentorship research opportunities, involvement in traditional conferences, and contributions to research journals. Part of this will be through promoting stewardship of open source scientific practices and “responsible” code release, as well as: supporting efforts that encourage ‘scientific communication’ through various media, such as: blog posts†
“WWe really want to shape Cohere For AI as an ambitious research lab that contributes to the research community, but also prioritizes how to better engage a diverse group of voices. We want to help change where, how and by whom research is done,” said Hooker.
Despite its lofty goals, Cohere For AI — which Cohere itself will fund — is likely to spark skepticism from researchers wary of Cohere’s corporate ties. Cohere has raised $170 million to date from institutional venture capital firms, including Tiger Global Management and Index Ventures, and has a number of ties to Google. Google Cloud AI chief scientist Fei-Fei Li and Google colleague Geoffrey Hinton were early supporters of Cohere, and Gomez and Frosst previously worked at Google Brain, one of Google’s AI research divisions. Cohere also has a partnership with Google to train large language models on the company’s dedicated hardware infrastructure.
Google infamous dissolved an AI advisory board in 2019, just a week after its creation. And in 2020, the company laid off Leading AI researcher Timnit Gebru in what she claimed was retaliation for sending colleagues an email critical of Google’s management practices. Google then fired another ethicist, Margaret Mitchell, who had publicly denounced the company’s handling of the situation, and a third, Satrajit Chatterjee, after he co-authored a paper describing Google’s work by AI powered chip design systems questioned.
Paved with good intentions
In general, nonprofit initiatives to fund AI research have been a mixed bag.
One of the success stories is The Allen Institute for AI (AI2), founded by the late Microsoft co-founder Paul Allen, which aims to achieve scientific breakthroughs by building AI systems with reasoning power. While not strictly non-profit, Anthropic, launched by former OpenAI executives, has raised more than half a billion dollars for research into “reliable, interpretable, and controllable” AI systems.
But for every AI2 and Anthropic there is an OpenAI, which started as a non-profit before moving on to capped profits and accepting a $1 billion investment from Microsoft. Meanwhile, former Google chairman Eric Schmidt’s recently announced $125 million fund for AI research drew new controversy after Politico. reported that Schmidt holds unusually heavy sway over the White House Office of Science and Technology Policy. (One of the first recipients, Berkeley professor Rediet Abebe, early to disregard her name.)
However, some newer collectives have shown promise, most notably Gebru’s Distributed AI Research, a global non-profit AI research organization. Projects like Hugging Face’s BigScience and EleutherAI are other strong examples of what can be achieved with AI beyond the confines of corporate influence.
“Ultimately, it’s up to us to prove that Cohere For AI is not going to become an enterprise over time,” Hooker said. “While Cohere For AI will rely on Cohere for resources and funding, a clear separation has been made between the two to maintain its independence as a research lab. This separation is crucial in order to continue to contribute to and serve the wider community as an independent entity. Cohere For AI is structured as a non-profit organization and is intentionally designed to openly collaborate with many different organizations. Her work will be open source to allow greater access to the wider community.”