Using AI to Make the World a Better Place

• The Haub School of Business at Saint Joseph’s University has teamed with Cabells Scholarly Analytics to create ChatSDG.
• The tool rates how well both scholarly journals and individual articles provide content that aligns with the Sustainable Development Goals.
• Schools can use reports generated by ChatSDG in their accreditation materials to demonstrate the impact created by their faculty research.

 
Two of the most talked-about trends in management education today are generative AI, as embodied by tools such as ChatGPT; and societal impact, particularly as measured by the United Nations Sustainable Development Goals (SDGs). Now these two trends have been combined in a pilot program based out of the Haub School of Business at Saint Joseph’s University in Philadelphia.

Working with Cabells Scholarly Analytics, the Haub School has developed ChatSDG, a large language model AI tool that provides a framework and metrics for measuring the impact of research on business and society. It not only is built upon the ethical standards embedded in the SDGs, but also satisfies AACSB’s accreditation requirement to show evidence of societal impact.

“Deans and faculty are constantly under pressure to deliver academic research that embraces rigor and relevance in equal measure,” says David Steingard, associate professor at the Haub School and director of the school’s SDG Dashboard initiatives. “Journal impact factors, citation counts, acceptance rates, editorial board composition, and journal rankings are conventional indicators of academic rigor. However, conceptualizing and measuring the relevance of research outside of academia, in the larger ecosystem of humanity and planet Earth, is much more challenging.”

ChatSDG, Steingard emphasizes, aims to make that challenge more manageable.

Assessing Impact

The new chatbot program builds on a previous joint project between Cabells and Saint Joseph’s. That project, the SDG Impact Journal Rating, evaluates the sustainability impact of the 50 journals the Financial Times uses to compile its FT Research Rank and 50 journals that publish in the general area of sustainability.

Using an algorithm that was trained on hundreds of thousands of articles, the rating measures each journal’s SDG Impact Intensity (SDGII) on a scale of 0 to 5, depending on how well it aligns with the SDGs. The SDGII also identifies the three SDGs that the publication focuses on most.

ChatSDG allows a school to see how many of its faculty’s articles are appearing in journals with high SDGII ratings; which three SDGs are featured most prominently in the school’s research; and how well its research aligns with the United Nations Agenda 2030, which lays out a plan for achieving the targets of the SDGs.

ChatSDG offers suggestions for how a school can increase the impact of its journal articles and how practitioners can use the research to advance the SDGs.

The chatbot then offers suggestions for how a school can increase the impact of its journal articles: how the research can be applied in public policy and business situations, how the research can be made widely available to practitioners, and how practitioners can use the research to advance the SDGs.

“Most academic research is written for other academics and is not readily accessible by practitioners,” Steingard points out. The report provides suggestions on “how to pinpoint real-world applications where faculty research can be operationalized.” It also identifies which existing stakeholders and champions—including agencies, advocacy groups, and policymakers—might be amenable to utilizing the research. This is important because “knowledge transfer is a much overlooked barrier to practitioner uptake of research,” says Steingard.

“These substantive and actionable recommendations move impact beyond scholar-to-scholar and toward scholar-to-society,” he adds. Armed with this information, schools can determine how their faculty research might be used to transform human, economic, and environmental welfare to solve “the vexing grand challenges of our time.”

How It Works

When a school submits its research to ChatSDG, the tool generates a customized, comprehensive report known as the AACSB Standard 8 Research Impact Report. Every report contains an interactive visualization platform based on the Haub School’s SDG Dashboard, where a school will find its SDG-related metrics and insights about individual research articles.

Because the report provides tangible metrics about the impact of the school’s research, the institution can include the report in the accreditation materials it presents to peer review teams. Therefore, the tool helps schools demonstrate that they are fulfilling AACSB’s accreditation Standard 8, which measures the impact of scholarship, and Standard 9, which considers societal impact.

While the data in each report is aggregated at the total faculty level, administrators can also use the tool’s interactive platform to assess the impact of individual articles—either before or after acceptance.

“We can think of ChatSDG as an additional reviewer who offers useful insights to enhance the rigor and relevance of scholarship,” says Steingard. Before publication, the tool can suggest ways an article can improve its alignment with the SDGs. After publication, ChatSDG can provide concrete suggestions for ways the article might be used by practitioners.

ChatSDG helps schools demonstrate that they are fulfilling the AACSB accreditation standards that consider the impact of scholarship.

Schools also can use ChatSDG’s reports to determine whether faculty are publishing in reputable outlets, because the tool incorporates Cabells’ journal quality designations. Publications listed in Cabells Journalytics Database have been fully vetted and meet the highest quality standards. Cabells categorizes other journals as Predatory (those found to be unethical or unlawful), Rejected (those that fail to meet Cabells’ standards of quality), or Pending Evaluation. The database also includes each journal’s article acceptance rate and its score in CiteScore Tracker, Elsevier’s journal impact factor rating.

“Surprisingly, most of the schools in ChatSDG’s pilot phase submitted at least a few articles to journals with the Predatory or Rejected designation,” says Steingard. “In turn, these low-quality or fraudulent journals were evidenced for Standard 8 documentation.” Schools that receive reports from ChatSDG will have “a verifiable method of guaranteeing that they only submit rigorous and relevant research for accreditation.”

Currently, ChatSDG is being used on a trial basis by a number of schools, but soon it will be more widely available to business schools. In fact, it will make its formal debut at AACSB’s 2024 International Conference and Annual Meeting to be held April 15–17 in Atlanta. While the Haub School of Business will generate reports for each school, Cabells will periodically publish aggregated data and overall insights from anonymized reports as a way of showing global trends.

“AI is unlike any other technology or trend I have seen in my three decades with AACSB. It will revolutionize the entire accreditation landscape,” says Joseph DiAngelo, dean of the Haub School and past chair of AACSB. “Our business schools will become more engaged and relevant ‘forces for good’ as a result of AI.”

An Impartial Judge

There’s already a working prototype powered by ChatSDG for articles. It’s the result of a partnership between the Haub School, Cabells, and the Responsible Research in Business and Management (RRBM) network. The RRBM SDG Impact Dashboard–Articles contains assessments of 238 journal articles from various disciplines that were recognized on the RRBM Honor Roll and as winners of discipline-based awards.

To evaluate what qualifies as “responsible research,” RRBM uses seven Principles of Responsible Science. These principles will be incorporated into future versions of the Standard 8 Research Impact Report and referred to as ChatSDG+RR7*.

“The RRBM Honor Roll and award evaluation initiative could be a major advancement toward defining, operationalizing, standardizing, and disseminating the fundamental principles of responsibility and impact of research,” says Steingard.

As part of the judging process, ChatSDG can screen entries at the start of the competition, replace human judges entirely, or function as a quality control measure after selections have been made.

In the future, ChatSDG will have a role in the judging process for the RRBM’s journal article honor roll and awards. Other organizations that sponsor sustainability competitions also could use ChatSDG as part of the judging process. “As long as there are clear standards of excellence and at least some evaluative history from human judges, ChatSDG can be programmed to learn the particulars of other competitions or applications,” Steingard says.

As part of the judging process, ChatSDG can be employed in several ways: It can screen entries at the start of the competition; it can replace human judges entirely; or it can be used after selections have been made as a quality control measure.

Steingard acknowledges that there might be some pushback from detractors who fear the “dark side” of AI and worry that human intelligence will be subordinated to machine intelligence. But in the case of awards judging, he thinks the benefits are clear.

“It seems that humans should be better at evaluating the societal impact of research,” he says. But once trained on RRBM’s Principles of Responsible Science, ChatSDG+RR7 “will likely prove to be more consistent, fair, efficient, and objective than human judges.”

What’s Ahead

While ChatSDG is still in its early stages, Steingard foresees a bright future. “I expect that, as with other key performance indicators, schools will use its reports to celebrate research impact when it exists and encourage more when it’s lacking,” he says.

Steingard believes that ChatSDG has great potential to promote sustainability across business and business curricula. Moreover, he hopes that the tool supports the creation of new AI-based technologies that tap “the power of academic research to better the human condition and sustain the earth.”

Note: SDG Dashboard, SDG Impact Intensity (SDGII), ChatSDG, and ChatSDG+RR7 are trademarks of Saint Joseph’s University.

For more information about how business schools are incorporating the SDGs into their curricula and their operations, check out these articles and publications: