I make plenty of errors. I fix some. Some I leave because imperfections make life more interesting. Imperfections allow us to change, adapt and evolve. My cicada blog Cicada Mania has plenty of errors. It has grammar errors, it has speculation and theorizing that deviates from fact, it has mobile usability issues Google demands that I fix, and it has old facts, once true, that are now false. I use Grammarly to check my grammar and spelling as I write. Often I find myself disappointed with its corrections because I feel it removes some of the flavor and improvisation of my writing. It kills the jazz. It deodorizes the funk. So I ignore it.
When looking into ChatGPT, I look at its responses through a human eye. I think I can see where its errors come from — they come from humans not tuning the AI and errors — like those mentioned above — that come from old and outdated sources. So, the answers it provides can be a little funky and a little wrong. I would not use ChatGPT to write a paper that needs to be 100% correct — maybe the outline of a paper and then do my own research, fill in the gaps and verify the data with multiple sources.
I tested ChatGPT‘s knowledge using a topic I’m very familiar with, “cicada species in New Jersey”. I hope to discover the limits of its knowledge and speculate where and how it fails to deliver a perfect answer. I encourage you to do the same with a topic you are very familiar with.
Findings:
- When it comes to cicadas (living insects), ChatGPT provides the binomial nomenclature, common name, and emergence frequency of each cicada. I was impressed with the inclusion of emergence frequency in its knowledge graph for cicadas, as emergency frequency is an important aspect of cicadas, whereas location, binomial nomenclature, and common name are applicable to all forms of life.
- ChatGPT is unaware of recent changes to the binomial nomenclature of many North American cicadas (Hill, 2015) (Sanborn, 2008). Perhaps it is learning from older publications & web pages — like a 15-year-old article on my website. My advice to ChatGPT would be to check recent documents when it comes to the binomial nomenclature of insects, and life in general.
- ChatGPT got several common names wrong. I imagine that this happens by learning from incorrect information on the web, getting information mixed on from a single web page, or when reading a category or tag page on a blog that cuts across multiple topics (in this case, multiple cicada species).
- ChatGPT put two cicadas in the list that are not found in New Jersey. The inclusion of Neotibicen superbus cicadas was way off — that cicada is located in and around Texas. The inclusion of Neotibicen pruinosus was respectable because prominent documents place Neotibicen pruinosus in New Jersey, but it is really its sister species Neotibicen winnemanna that is found in New Jersey, with Neotibicen pruinosus existing west of the Appalachians.
- I would give ChatGPT an overall grade of C. All the lifeforms in its lists are cicadas, 80% are found in New Jersey,
Remarking on ChatGPT’s response to “List of cicada species in New Jersey” line by line:
Cicadas are a group of insects known for their loud mating calls. There are several species of cicadas that can be found in New Jersey, here is a list of some of them:
So far, ChatGPT is correct. Grade A.
1. Magicicada septendecim: known as the “17-year cicada” it emerges in large numbers every 17 years.
- Correct location.
- Correct binomial nomiclature.
- Correct common name.
- Correct emergence frequency.
Grade: A+ (100/100).
Trivia: Magicicada septendecim is also known as the Pharoah Cicada.
2. Magicicada cassini: known as the “13-year cicada” it emerges in large numbers every 13 years.
- Correct location.
- Correct binomial nomiclature.
- Incorrect common name. “17-year cicada” and “Dwarf Cicada” would be the correct answers.
- Incorrect emergence frequency. Magicicada cassini emerges every 17 years.
Grade C- (70/100).
Trivia: There is a similar species named Magicicada tredecassini that has a 13-year lifecycle, but it is not found in New Jersey. More information about Magicicada cassini.
3. Tibicen canicularis: known as the “dog-day cicada” it emerges every year.
- Correct location.
- Incorrect binomial nomiclature. Tibicen canicularis has been known as Neotibicen canicularis since 2015 (see Hill 2015).
- Correct common name.
- Correct emergence frequency.
Grade: A- (90/100).
Trivia: more info about Neotibicen canicularis.
4. Tibicen chloromera: known as the “green-faced cicada” it emerges every year.
- Correct location.
- Incorrect binomial nomiclature. Incorrect genus (Hill, 2015) and species (Sanborn, 2008).
- Possibly a correct common name. Popular common names for Neotibicen tibicen are “Swamp Cicada” and “Morning Cicada”. I’ve never heard of the name “green-faced cicada”, but I do not have evidence to the contrary, so I cannot dispute this. Interestingly enough, chloromera comes from the Greek words meaning “green thighs”. Certainly both its legs and face can be green.
- Correct emergence frequency.
Grade B- (80/100).
5. Tibicen linnei: known as the “Linne’s cicada” it emerges every year.
- Correct location.
- Incorrect binomial nomiclature. Tibicen linnei has been known as Neotibicen linnei since 2015 (see Hill 2015).
- Correct common name.
- Correct emergence frequency.
Grade A- (90/100).
6. Tibicen tibicen: known as the “harvest fly” it emerges every year.
- Duplicate entry!
- Incorrect binomial nomiclature. Tibicen tibicen has been known as Neotibicen tibicen since 2015 (see Hill 2015). Note there are two subspecies.
- Correct location.
- Debatable common name. Harvest fly, Dog-Day Cicada, or Jar Fly, are common names for all Neotibicen and Megatibicen cicadas in general, but Swamp Cicada and Morning Cicada are the names popularly used for this cicada.
- Correct emergence frequency.
Grade C- (70/100).
7. Tibicen lyricen: known as the “northern dusk-singing cicada” it emerges every year.
- Correct location.
- Incorrect binomial nomiclature. Tibicen lyricen has been known as Neotibicen lyricen since 2015 (see Hill 2015).
- Incorrect common name. The correct common name for this cicada is Lyric Cicada. There is a “northern dusk-singing cicada” in New Jersey, but that species is Megatibicen auletes.
- Correct emergence frequency.
Grade C (75/100).
8. Neotibicen pruinosus: known as the “pruinose cicada” it emerges every year.
- Incorrect location. This cicada is not found in New Jersey. Its very close relatives Neotibicen winnemanna and Neotibicen latifasciatus are found in New Jersey. Notes on locations from BugGuide.net.
- Correct binomial nomiclature.
- Correct common name.
- Correct emergence frequency.
Grade: D- (60/100).
9. Neotibicen superbus: known as the “superb cicada” it emerges every year.
- Incorrect location. This cicada is not found in New Jersey. It is found in Texas and surrounding states.
- Correct binomial nomiclature.
- Correct common name.
- Correct emergence frequency.
Grade: D- (60/100).
10. Okanagana rimosa: known as the “eastern cicada” it emerges every year.
- Correct location.
- Correct binomial nomiclature.
- Possibly a correct common name. The popular common name for this cicada is Say’s Cicada. I’ve never heard of “eastern cicada” applied to this species, but it may be correct.
- Correct emergence frequency.
Grade: A- (90/100).
It’s worth mentioning that this list is not exhaustive and there may be other species of cicadas found in New Jersey, also the emergence of cicadas is affected by weather, temperature, and the health of the population, so the emergence of some species may vary from year to year.
Correct.
Grade: A.
Again, overall ChatGPT gets a C.
Grading scale:
- Is the location correct (New Jersey)? 40 points.
- Is it a cicada? 15 points.
- Is the binomial nomenclature correct? 15 points.
- Was the binomial nomenclature correct at one time? 5 points.
- Is the common name the most popular common name? 15 points.
- Is the common name a lesser-known common name? 5 points.
- Is the emergence frequency correct? 15 points.
- Duplicate entry penalty: -5 points.
- 2015 Neotibicen cicada name change document: Hill KB, Marshall DC, Moulds MS, Simon C. Molecular phylogenetics, diversification, and systematics of Tibicen Latreille 1825 and allied cicadas of the tribe Cryptotympanini, with three new genera and emphasis on species from the USA and Canada (Hemiptera: Auchenorrhyncha: Cicadidae). Zootaxa. 2015 Jul 10;3985(2):219-51. doi: 10.11646/zootaxa.3985.2.3. PMID: 26250031.
- 2008 chloromera to tibicen name change document: Allen F. Sanborn “The Identity Of Cicada tibicen Linné [=Tibicen chloromerus ()] (Hemiptera: Cicadoidea: Cicadidae),” Entomological News 119(3), 227-231, (1 May 2008). https://doi.org/10.3157/0013-872X(2008)119[227:TIOCTL]2.0.CO;2
