At the Frankfurt Book Fair this week, a startup company based in Toronto has claimed to have created artificial intelligence that out-performed their scientific editors in recognizing and selecting manuscripts. The company, Meta, has titled their innovative product Bibliometrics Intelligence, and Aries Systems Corporation has integrated it into their manuscript and peer review tracking system, Editorial Manager.

According to the release, “Large-scale trials conducted by Meta in partnership with industry demonstrated that Bibliometric Intelligence out-performed tens of thousands of human editors by a factor 2.5x at predicting article-level impact for new manuscripts, prior to publication. It also performed 2.2x better than the same group of editors at identifying “superstar articles” – those that represent the top 1% of high-impact papers, prior to publication.”

Meta claims the program utilizes machine learning algorithms that were trained through their collection of millions of articles. They now assert that when new manuscripts are processed in the program, unique features are pulled from the text, put into their algorithms, and then it “estimates the future citation count and impact of a manuscript, with a speed, accuracy, and consistency that far exceed human ability”.

If this is the case, Bibliometrics Intelligence is an extremely daunting disruptive force about to hit the scholarly scientific publishing community. The editing process could be completely changed, and the amount of editors needed for publications would be diminished.

However, Phil Davis writes in his article for Scholarly Kitchen that when he tried to reach out to the two companies for a copy of their study, so he could read the detailed results, he was denied because the company was in the process of publishing the results. Davis insists that this violates the Ingelfinger rule and without results/document there is no substantial claim.

Either way, it seems we are not far from artificial technology taking parts of the editorial and publishing process. Is it possible that we will create artificial intelligence that is capable of judging all qualities of scholarly publishing? Will robots take over the whole business? Who’s to know? But, I imagine for some, this disruptive force is extremely scary…


Can An Algorithm Outperform Science Editors?

Artificial Intelligence Integration Allows Publishers a First Look at Meta Bibliometric Intelligence