Comparing and Testing AI for Education

AI robots becoming the new rulers, a grand throne room filled with robots in regal attire, adorned with glowing symbols and intricate metalwork, human ambassadors kneel in submission, the mood is one of awe and submissiveness, Artwork, a detailed Renaissance-style oil painting with the use of dramatic chiaroscuro to highlight the metallic sheen and grandeur of the robots

Professor and friend John Nash co-hosts a podcast on all things online learning. In a recent episode, he shared his work on coaching ChatGPT to write more “human” and the results are… interesting…

While generative AI tools are very cool right now, they are a long way from being truly disruptive and overtaking the world.

Here’s what’s interesting. Scaffolding the prompts, defining perplexity and burstiness, and then prompting an explicit increase of those measures made the text “human” to GPTZero. Still, it also made the text ridiculously flowery and inflated. Kind of like when a master’s student thinks they are supposed to “sound academic.” It was so bad that the ChatGPT output was immediately suspect to my human eyes, even though GPTZero said it was likely written entirely by a human.

– John Nash, PhD