Educational Materials

What Fraction of Possible Letter Sequences Are Words?

Stephen Wolfram

Author

Stephen Wolfram

Title

What Fraction of Possible Letter Sequences Are Words?

Description

Get the lengths of words in the word list for English:

Category

Educational Materials

Keywords

URL

http://www.notebookarchive.org/2019-08-98zxh94/

DOI

https://notebookarchive.org/2019-08-98zxh94

Date Added

2019-08-20

Date Last Modified

2019-08-20

File Size

49.77 kilobytes

Supplements

Rights

Redistribution rights reserved

What Fraction of Possible Letter Sequences Are Words?

Get the lengths of words in the word list for English:

In[]:=

StringLength[WordList[]]

Out[]=

{1,3,8,5,6,5,7,7,9,11,5,9,5,7,9,5,9,8,4,6,5,5,10,11,12,8,10,7,9,6,9,9,8,5,4,8,10,4,7,8,5,10,9,8,5,7,7,6,9,8,10,6,7,6,7,8,8,6,4,6,8,4,8,10,8,11,10,6,5,

⋯39989⋯

,12,4,9,7,4,5,5,3,5,6,4,4,3,6,4,6,8,7,9,5,4,3,6,6,8,4,6,4,7,9,11,5,4,6,6,5,7,4,9,4,6,6,3,6,5,6,9,3,6,5,6,8,6,5,4,6,3,10,9,7,4,8,6,8,8,6,6,7,7}

large output

show less

show more

show all

set size limit...

Make a histogram of their lengths:

In[]:=

Histogram[%]

Out[]=

There is only one word of length 1 in the word list:

In[]:=

Select[WordList[],StringLength[#]1&]

Out[]=

{a}

Here are the words of length 2:

In[]:=

Select[WordList[],StringLength[#]2&]

Out[]=

{ad,ah,an,as,at,ax,be,by,dB,do,eh,em,en,er,ex,fa,go,ha,he,hi,hm,ho,id,if,in,it,kc,kW,la,lo,ma,me,mi,ms,mu,my,no,nu,of,oh,on,ow,ox,pa,pH,pi,re,sh,so,ta,ti,to,um,up,us,we,xi,ya,ye}

(Some of them are slightly questionable words...)

Here is a count of the number of length-2 words in the word list:

In[]:=

Length[Select[WordList[],StringLength[#]2&]]

Out[]=

59

Here is a list of all possible pairs of letters:

In[]:=

StringJoin/@Tuples[Alphabet[],2]//Short

Out[]//Short=

{aa,ab,ac,ad,ae,af,ag,ah,660,zs,zt,zu,zv,zw,zx,zy,zz}

Find the number of possible pairs:

In[]:=

Length[%]

Out[]=

676

One can compute the number of possible pairs like this:

In[]:=

26^2

Out[]=

676

Note that the number of actual 2-letter words is much smaller than the number of combinations of 2 letters.

The list of possible 3-letter combinations:

In[]:=

StringJoin/@Tuples[Alphabet[],3]

Out[]=

{aaa,aab,aac,aad,aae,aaf,aag,aah,aai,aaj,aak,aal,aam,aan,aao,aap,aaq,aar,aas,aat,aau,aav,aaw,aax,

⋯17528⋯

,zzc,zzd,zze,zzf,zzg,zzh,zzi,zzj,zzk,zzl,zzm,zzn,zzo,zzp,zzq,zzr,zzs,zzt,zzu,zzv,zzw,zzx,zzy,zzz}

large output

show less

show more

show all

set size limit...

A count of them:

In[]:=

26^3

Out[]=

17576

Here is the number of actual 3-letter words in the word list:

In[]:=

Length[Select[WordList[],StringLength[#]3&]]

Out[]=

594

The count of possible 4-letter sequences:

In[]:=

26^4

Out[]=

456976

The actual number of 4-letter words:

In[]:=

Length[Select[WordList[],StringLength[#]4&]]

Out[]=

1951

Things to do

Find the Shannon entropy by fitting 26^(h n). Look at the results for different human languages.

Cite this as: Stephen Wolfram, "What Fraction of Possible Letter Sequences Are Words?" from the Notebook Archive (2019), https://notebookarchive.org/2019-08-98zxh94