Accessing Monarch Biodiversity Data with New Wolfram Language Functions
Accessing Monarch Biodiversity Data with New Wolfram Language Functions
Author
Jofre Espigule-Pons
Title
Accessing Monarch Biodiversity Data with New Wolfram Language Functions
Description
Learn how to document & preserve biodiversity using Wolfram Language data access functions in the Function Repository; join community of citizen scientists from iNaturalist mapping species geography, classifying specimens, studying biotic interactions & more.
Category
Essays, Posts & Presentations
Keywords
Developer Insights, Function Repository, Recreational Computation
URL
http://www.notebookarchive.org/2020-10-2taa8j4/
DOI
https://notebookarchive.org/2020-10-2taa8j4
Date Added
2020-10-06
Date Last Modified
2020-10-06
File Size
33.07 megabytes
Supplements
Rights
Redistribution rights reserved
Download
Open in Wolfram Cloud
October 6, 2020 — Jofre Espigule-Pons, Machine Learning
Earth has experienced five major extinctions since life first appeared almost four billion years ago. The sixth is happening right now; the current extinction rate is between one hundred and one thousand times greater than what it was before 1800.
Despite the alarming extinction rate, it’s easier than ever to document biodiversity with the help of the Wolfram Language. Using the monarch butterfly as an example, I will explore the new biodiversity data access functions in the Wolfram Function Repository and how they can help you join a community of thousands of citizen scientists from iNaturalist in preserving biodiversity.
I developed a set of biodiversity data access functions—INaturalistImport, INaturalistSearch, GBIFImport, GBIFSearch and GloBIData—to better enable researchers, citizen scientists, educators and Wolfram users to access global biodiversity data from the iNaturalist community, Global Biodiversity Information Facility (GBIF) and Global Biotic Interactions (GloBI).
Becoming a Citizen Scientist Crowdsourced data and observations from citizen scientists can be valuable for researchers, allowing global communities to track biodiversity worldwide. Over the last year, thanks to the iNaturalist app, I observed and identified hundreds of different species during hikes and got invaluable feedback from many different experts, each specializing in some particular life form. There are currently more than 290,000 different species recorded on the platform, a handful of which I was the first iNaturalist user to observe.
A great starting place as a beginner citizen scientist is to learn more about the species you want to find. If you know the scientific name—for the monarch butterfly, it’s Danaus plexippus— allows you to better visualize what you’re looking for:
WikidataSearch
In[]:=
WikidataSearch["Danaus Plexippus"]
Out[]=
,,,,
In[]:=
Out[]=
I can also check for some more specific properties, like wingspan, with :
WikidataData
In[]:=
Out[]=
{}
90.
mm
Or ask for their maximum speed using its species entity ( + =):
In[]:=
monarch butterfly | » | |||
| ||||
Out[]=
Anosia plexippus
Note that Anosia is a synonym name for the taxon of the genus Danau:
In[]:=
Anosia plexippus | SPECIES SPECIFICATION |
maximum speed
Out[]=
18000000
mm/h
Let’s convert the units into more familiar speed formats:
In[]:=
UnitConvert[,]
18000000
mm/h
km/h
Out[]=
18
km/h
In[]:=
N@UnitConvert[,]
18000000
mm/h
mi/h
Out[]=
11.1847
mi/h
Mapping Species Geography Monarch butterflies are one of the most amazing insects, capable of flying thousands of kilometers. The majority of them overwinter in a specific region of Mexico and fly back toward the north of North America in four generations (around a two-month period). During the end of summer when the sunlight hours decrease and temperatures drop, a different generation of monarch flies all the way down to Mexico in a single generation.
Using INaturalistSearch, I can easily create a map with all of the North American iNaturalist observations since yesterday:
In[]:=
ResourceFunction["INaturalistSearch"]
Out[]=
|
In[]:=
monarchData=
"Danaus plexippus","ObservationGeoRange"GeoBounds
,"ObservationDateRange"{Yesterday,Today},"QualityGrade""Research";
|
North America | GEOGRAPHIC REGION |
The output is a large Dataset; we can check the first observation, for example:
In[]:=
First@monarchData
Out[]=
| |||||||||||||||||||||||||||||||||||||||||||
Let's use a custom GeoMarker icon image to represent each recorded observation of the monarch butterfly:
In[]:=
monarchIcon=Rotate
,-30Degree;
In[]:=
GeoGraphics{GeoMarker[monarchData[All,"GeoPosition"],monarchIcon,"Scale"0.06]},ImageSize500,GeoRangeGeoBoundsPolygon
,
United States | COUNTRY |
Mexico | COUNTRY |
Out[]=
Next, I would like to visualize their annual migration. But I want to first check the total count of observations over a year (2019), because if there are too many (over one thousand observations), then we should avoid using INaturalistSearch multiple times (See the iNaturalist API recommended practices page for details):
In[]:=
|
North America | GEOGRAPHIC REGION |
Out[]=
33113
There are over thirty thousand observations! So it is clear that I will need to use either INaturalistImport or GBIFImport.
Now that we have the file, we are ready to import the data using GBIFImport:
In[]:=
monarchData=
["NorthAmericaOccurrences.csv"]//Quiet;
|
GBIF gathers data from multiple institutions; by using on the "InstitutionCode" property, we can see that the vast majority of observations on monarch butterflies are coming from iNaturalist users:
Counts
In[]:=
Counts@monarchData[All,"InstitutionCode"]
Out[]=
|
We can create a to see when there are more observations of monarchs:
DateHistogram
In[]:=
observationDates=monarchData[All,"Date"];
In[]:=
DateHistogram[observationDates,"Day",ColorFunction"SolarColors",ImageSizeLarge,PlotTheme"Detailed"]
Out[]=
Not surprisingly, observations rise in March when the northward migration starts, and peak in August when monarch butterflies are highly present in gardens of the northern US—where the density of iNaturalist users is higher.
As a fun experiment, we can compare daily page hits for two related Wikipedia articles (“Monarch butterfly” and “Monarch butterfly migration”) using and :
WikipediaData
DateListLogPlot
In[]:=
monarchButterfly=WikipediaData["Monarch butterfly","DailyPageHits"];monarchMigration=WikipediaData["Monarch butterfly migration","DailyPageHits"];
In[]:=
DateListLogPlot[{monarchButterfly,monarchMigration},PlotRange{{DateObject[{2019}],DateObject[{2019,12,31}]},All},PlotLegends{"Monarch butterfly","Monarch butterfly migration"},ImageSize600,PlotTheme"Detailed"]
Out[]=
 |
|
The “Monarch butterfly” article gets a higher number of daily visits during July and August when the presence of monarchs is higher in the US. We can also observe two peaks for the “Monarch butterfly migration” article corresponding to the start of northward migration in March and another in September when they fly southwest toward Mexico.
It seems that people search for info about the butterflies when they hear about the start of the northward migration, but it isn’t until summer when they actively go out and record their observations of monarchs.
If we compare this with page views of the Spanish version of the article, the graph shows a decrease during summer when monarchs are mainly in the US:
In[]:=
monarchButterfly=WikipediaData"Monarch butterfly","DailyPageHits",Language
;
English | LANGUAGE |
Spanish | LANGUAGE |
In[]:=
DateListLogPlot[monarchButterfly,PlotRange{{DateObject[{2019}],DateObject[{2019,12,31}]},All},PlotLegends{"Monarch butterfly (Spanish page)"},ImageSize600,PlotTheme"Detailed"]
Out[]=
 |
|
To create the map animation of the migration, we need to group the observations by date. This is easily achieved in the dataset using :
GroupBy
In[]:=
observationsPerDay=monarchData[GroupBy["Date"]];
Next, we need to create some plotting utilities to make a more interesting map:
In[]:=
colorDateFunction[date_DateObject]:=ColorData["BrightBands"][ToExpression@DateValue[date,"ISOYearDayShort"]/365]
The previous function will allow us to represent observations on the map with different colors that correspond to their observation dates. We can create its corresponding legend using :
In[]:=
legend=SwatchLegend[Map[ColorData["BrightBands"][(#-0.5)/12]&,Range[12]],Map[Style[#,FontSize18]&,{"January","February","March","April","May","June","July","August","September","October","November","December"}],LegendFunction(Framed[#,RoundingRadius5,FrameStyleWhite,BackgroundWhite]&),LegendLayout{"Column",2},LegendMarkerSize15,LegendMarkers"Bubble"]
Out[]=
|
|
We also need to get the geographic positions associated with each day:
In[]:=
getGeoPositionsPerDate[date_DateObject]:=KeyTake[observationsPerDay,date][[1,All,"GeoPosition"]]
The above one-liner will get a list of GeoPosition objects corresponding to the observations of a specific date.
The following function will get a list of all days since the start of 2019 until the specified date:
In[]:=
getDates[date_DateObject]:=DateRange[DateObject[{2019,1,1}],date]
Finally, we can add some random tumbling to the monarch icons using and , to give them a little life and make them look like they’re flying:
Rotate
RandomReal
In[]:=
mapNorthMigrationByDate[date_]:=Rasterize@GeoGraphics[Join[Flatten@Map[{colorDateFunction[#],Opacity[0.8],PointSize[.009],Point[getGeoPositionsPerDate[#]]}&,getDates[date]],{Map[GeoMarker[#,Rotate[monarchIcon,RandomReal[{-20,20}]Degree],"Scale"0.08]&,getGeoPositionsPerDate[date]]},{Black,Inset[Framed[Style["Observation Date\n"<>DateString[date,"ISODate"],FontFamily"Helvetica",FontSize22],FrameStyleWhite,RoundingRadius4,BackgroundWhite],Scaled[{0.11,0.88}]],Black,Inset[legend,Scaled[{0.15,0.2}]]}],ImageSize{1010,480},GeoRange{{14,48},{-124,-67}},GeoBackground"StreetMapNoLabels"]
Let’s try a map for a particular day:
In[]:=
mapNorthMigrationByDate[DateObject[{2019,4,17}]]
Out[]=
We can do the same for the migration toward the south that starts in September by rotating the icons top to bottom:
In[]:=
mapSouthMigrationByDate[date_]:=Rasterize@GeoGraphics[Join[Flatten@Map[{colorDateFunction[#],Opacity[0.8],PointSize[.009],Point[getGeoPositionsPerDate[#]]}&,getDates[date]],{Map[GeoMarker[#,Rotate[monarchIcon,RandomReal[{-170,-210}]Degree],"Scale"0.08]&,getGeoPositionsPerDate[date]]},{Black,Inset[Framed[Style["Observation Date\n"<>DateString[date,"ISODate"],FontFamily"Helvetica",FontSize22],FrameStyleWhite,RoundingRadius4,BackgroundWhite],Scaled[{0.11,0.88}]],Black,Inset[legend,Scaled[{0.15,0.2}]]}],ImageSize{1010,480},GeoRange{{14,48},{-124,-67}},GeoBackground"StreetMapNoLabels"]
Cool! Now we only need to create one map for each day of the year and use to obtain an APNG of each one (note that it will take a while to create that many frames with 22,000 observations):
Export
In[]:=
Export["MonarchMigration.apng",Join[Map[mapNorthMigrationByDate,DateRange[DateObject[{2019,1,1}],DateObject[{2019,8,31}]]],Map[mapSouthMigrationByDate,DateRange[DateObject[{2019,9,1}],DateObject[{2019,12,31}]]]],"DisplayDurations"0.5,"AnimationRepetitions"Infinity]
In[]:=
Classifying Male and Female Specimens Let’s try some machine learning techniques on monarch butterflies. The following photo was taken by one of my colleagues, Cassidy, who has a beautiful garden in Champaign, Illinois, with many milkweed plants, the feeding source for monarch caterpillars:
In[]:=
In[]:=
ImageIdentify
,
,10,"Probability"
insects | SPECIES SPECIFICATION |
Out[]=
0.987214,0.987214,0.990018,0.990024,1.,0.00242406,0.00279069,0.00921452,0.00921452
monarch butterfly
danaid butterfly
butterfly
lepidopterous insect
insect
viceroy
brush-footed butterfly
chrysalis
pupa
Some iNaturalist observations have annotations attached. For example, users can annotate a specimen’s sex as female or male. This info might be useful to scientists studying the ratio of females to males. But in reality, only a small percentage of users specify whether the monarch they observed was a male or a female. If we could automate these manual annotations of a specimen’s sex, we could create a much larger dataset in order to study female vs. male populations. Let’s try to gather images of monarch females and males and train our own sex classifier with :
Classify
In[]:=
males=MapImport,Normal
["Danaus plexippus","Sex""Male","HasImage"True,"QualityGrade""Research"][All,"SquareImageURL"]//Quiet;
|
In[]:=
females=MapImport,Normal
["Danaus plexippus","Sex""Female","HasImage"True,"QualityGrade""Research"][All,"SquareImageURL"]//Quiet;
|
Create a dataset with a randomized list of key values:
In[]:=
dataset=RandomSample@Flatten[Thread/@Thread[{males,females}{"Male","Female"}]];
In[]:=
dataset[[1]]
Out[]=
Separate the dataset into a training set and a test set:
In[]:=
trainingSet=dataset[[;;150]];testSet=dataset[[151;;]];
Then we can create a classifier using Classify:
In[]:=
c=Classify[trainingSet,PerformanceGoal"Quality"]
Out[]=
ClassifierFunction
| |||||
Data not saved. Save now |
The resulting classifier seems to have a fairly high accuracy of around 75%. For example, I don’t know how to distinguish male from female monarchs without having a reference:
In[]:=
Information[c]
Out[]=
Classifier information | ||||||||||||||||||||||
| ||||||||||||||||||||||
Now we can easily plot the confusion matrix of the classifier using the test set:
In[]:=
cm=ClassifierMeasurements[c,testSet,"ConfusionMatrixPlot"]
Out[]=
Nice! Let’s try a couple of specific examples:
In[]:=
testSet[[2]]
Out[]=
In[]:=
c
Out[]=
Male
In[]:=
testSet[[7]]
Out[]=
In[]:=
c
Out[]=
Female
It seems to be working decently well with stock images. Let’s try using a photo of a monarch taken by my friend Cassidy in her garden. She told me it was a female—will our classifier confirm her guess?
In[]:=
c,"Probabilities"
Out[]=
Female0.581465,Male0.418535
You could try to improve the current classifier by using more images or data augmentation techniques and . You can start with a neural net model from the Wolfram Neural Net Repository used for image classification problems, like the Squeeze-and-Excitation Net or the Wolfram ImageIdentify Net V1.
Exploring Biotic Interactions Another way to help you document species is to study what they eat so you know where to find them. Let’s take a closer look at the monarch butterfly’s diet. The Global Biotic Interactions platform has several gigabytes of data containing millions of interactions among species extracted from scientific papers, the Encyclopedia of Life (EOL) and research-grade observations from iNaturalist and GBIF. Here we will explore the food web of monarchs, the animals that feed on milkweed plants (the main diet of monarch caterpillars) and finally, potential monarch parasites:
In[]:=
ResourceFunction["GloBIData","Description"]
Out[]=
Gives biotic interactions of a specified taxon using the Global Biotic Interactions (GloBI) API
On the first argument of GloBIData, we need to specify the taxon. As an option, we can retrieve specific interaction types:
In[]:=
|
Out[]=
Wow, that’s a huge food web! Its diet contains many species of plants, including a milkweed (genus Asclepias) that larvae feed on and that also appears in the left cluster. Let’s take that information and use it to find out what the milkweed looks like:
In[]:=
Asclepias | » | |||
| ||||
In[]:=
milkweed | PLANT |
Out[]=
Last year, during the 2019 Wolfram Summer School in Waltham, Massachusetts, I found a common milkweed during a forest walk near Bentley University. Importing my photo to help other citizen scientists find milkweed—and perhaps monarchs feeding on it—is now straightforward with INaturalistSearch:
In[]:=
ImageResize[Import@First[INaturalistSearch["Asclepias syriaca","UserLogin""jofree"][All,"ImageURL"]],600]
Out[]=
Monarch butterflies have high concentrations of a toxic molecule extracted from milkweed plants. This prevents them from being eaten by many potential predators. If a predator eats them, the toxins cause it to get sick or even suffer cardiac arrest.
With GloBIData, I can also do the reverse and ask for moths and butterflies feeding on milkweed plants by specifying a "TargetTaxon":
In[]:=
|
milkweed | PLANT |
moths and butterflies | SPECIES SPECIFICATION |
Out[]=
Although the monarch larvae are the main consumers of milkweed, many other types of butterflies and moths also like it.
I can also get a taxonomic graph of organisms that are parasites of monarchs using the "HasParasite" interaction type:
In[]:=
|
Out[]=
Monarch Butterfly Biosphere Reserve During winter, monarchs travel by the thousands to mountain forests in Mexico. One of the most famous wintering sanctuaries is the Monarch Butterfly Biosphere Reserve, a World Heritage Site in Michoacán, which is about 60 miles northwest of Mexico City. Let’s find the exact location using WikidataSearch:
In[]:=
WikidataSearch["monarch butterfly biosphere"]
Out[]=
In[]:=
winteringLocation=First@WikidataData,;
We can easily see the massive distance that, for example, Cassidy’s monarch (born in Champaign) will need to fly:
In[]:=
GeoDistance
,winteringLocation
Champaign | CITY |
Out[]=
2537.82
km
We can also plot the trip with and :
GeoPath
Arrow
In[]:=
GeoGraphicsRed,Thick,Arrow@GeoPath
,winteringLocation,ImageSize600,GeoBackground"Satellite",GeoRangePolygon
,
Champaign | CITY |
United States | COUNTRY |
Mexico | COUNTRY |
Out[]=
The Wolfram Language is also capable of getting satellite images. For example, we can look at the Biosphere Reserve using :
GeoImage
In[]:=
GeoImage[winteringLocation,GeoServer"DigitalGlobe"]
Out[]=
As a final exploration, we can also check the observations of threatened species near the previous location using and INaturalistSearch:
GeoDisk
In[]:=
threatenedSpecies=
[All,"ObservationGeoRange"GeoBounds[GeoDisk[winteringLocation,]],"Threatened"True,"QualityGrade""Research","HasImage"True];
|
3km
For a more visual understanding, we can create a of the different types of threatened life forms from the previous list using the "IconicTaxon" property and Counts:
PieChart
In[]:=
PieChartCounts@Normal@threatenedSpecies[All,"IconicTaxon"],
Out[]=
 |
|
We can see that the majority of observations are from insects, monarchs in particular. But there are some mushrooms, plants and even a bird and an amphibian counted as threatened species.
Similar to our visual pie chart, an image is worth a thousand words. So let’s create a final out of all the observations of threatened species from the forest sanctuary:
ImageCollage
In[]:=
ImageCollage@Map[Import,Normal@threatenedSpecies[All,"SquareImageURL"]]
Out[]=
Computing a More Biodiverse Future I hope you enjoyed these explorations. The number of applications that these new biodiversity data access functions open up is limited only by our imaginations, especially if we start combining them with the variety of built-in Wolfram Language functions. For example, use to automatically alert you via email with to the presence of an invasive species in a certain region or new sightings of endangered species in a forest with risk of deforestation.
ScheduledTask
SendMail
If you’re looking for more ideas on some of the different ways you can use the Wolfram Language in your adventures, check out my last blog post on the invasive stink bug, as well as my Wolfram Community posts on marine worms and Moon phases and the great horned owl.
Finally, I want to thank Wolfram Research programmer John Cassel for his valuable advice during my journey to bring these functions to life. I would also like to thank professor Gareth Russell of the Department of Ecology at the New Jersey Institute of Technology for his valuable feedback on the functions. The ability to directly import GBIF data is an important step toward the implementation of species distribution modeling using the Wolfram Language. These models are widely used in ecology, and they help to monitor changes in biodiversity and, consequently, help to protect our marvelous planet.
The golden age for becoming a citizen scientist is now. Humanity has counted almost two million distinct life forms—only a small fraction of what scientists believe is still waiting to be found. How many will you find?
Cite this as: Jofre Espigule-Pons, "Accessing Monarch Biodiversity Data with New Wolfram Language Functions" from the Notebook Archive (2020), https://notebookarchive.org/2020-10-2taa8j4
Download
