Essays, Posts & Presentations

Face Analysis from Star Trek Video

Wolfram Research, Inc.

Author

Wolfram Research, Inc.

Title

Face Analysis from Star Trek Video

Description

This example shows how to leverage machine learning and video processing functionality to identify the faces of a set of of characters in a Star Trek promotional video.

Face Analysis from Star Trek Video

This example shows how to leverage machine learning and video processing functionality to identify the faces of a set of of characters in a Star Trek promotional video.

Curating a dataset of faces of Star Trek characters can be automated using high level functions such as WebImageSearch and FindFaces.

In[]:=

searchCharacterPhotos[name_]:=Flatten[FindFaces[#,"Image"]&/@WebImageSearch["star trek "<>name]]faces=AssociationMap[searchCharacterPhotos,{"Jean-Luc Picard","William Riker","Phillipa Louvois","Data"}];

Display thumbnails for the result of the search.

In[]:=

Map[ImageResize[#,30]&,faces,{2}]

Out[]=

Jean-Luc Picard

,William Riker

,Phillipa Louvois

,Data



Import a Star Trek promotional video:

In[]:=

starTrekVideo=Video[CloudObject[

https://www.wolframcloud.com/obj/documentation/StarTrekPromo

]]

Out[]=

Advanced machine learning functionality allows for the creation of a powerful face recognizing function even if the provided dataset is small. Use FaceRecognize to find and identify faces in an image.

In[]:=

FaceRecognize[faces,VideoExtractFrames[starTrekVideo,11.2],All]//Dataset

Out[]=

Image	BoundingBox	Name	Probability	RarerProbability	Position
	Rectangle[{410.3,245.6},{508.7,376.4}]	Phillipa Louvois	0.934524	1	{459.5,311.}
	Rectangle[{82.4,271.8},{203.6,412.2}]	Jean-Luc Picard	0.934524	1	{143.,342.}

FaceRecognize relies on neural networks and transfer learning to automatically train a ClassifierFunction.

In[]:=

recognizer=FaceRecognize[faces]

Out[]=

ClassifierFunction

Input type: Image

Classes: Data,Jean-Luc Picard,Phillipa Louvois,William Riker



Create a simple function to massage the output of the face recognizer into something easier to plot.

In[]:=

recognizeFaces[a_]:=Thread@{a["Time"],ReplaceAll[recognizer[FindFaces[a["Image"],"Image"]],{{}->{None},_Missing->"Other"}]}

Apply the recognizer function on all the frames of the video.

In[]:=

recognizedFaces=Join@@VideoMapList[recognizeFaces,starTrekVideo];

Aggregate and reduce the result.

In[]:=

recognizedFaces=GroupBy[DeleteCases[recognizedFaces,{_,None}],Last,Map[First]];

Plot the resulting timeline.

In[]:=

NumberLinePlot[recognizedFaces,PlotLegendsKeys[recognizedFaces]]

Out[]=

	William Riker
	Phillipa Louvois
	Jean-Luc Picard
	Data
	Other

Cite this as: Wolfram Research, Inc., "Face Analysis from Star Trek Video" from the Notebook Archive (2022), https://notebookarchive.org/2022-07-bka73cl