Educational Materials

04 Structure Search

Joshua Schrier

Author

Joshua Schrier

Title

04 Structure Search

Description

Objectives

Structure Search

NOTE: This is a 2 week assignment

Objectives

◼

Learn various types of structure searches including identity search, similarity search, substructure and super structure searches.

◼

Learn the optional parameters available for each search type.

Using PUG-REST, one can perform various types of structure searches (https://bit.ly/2lPznCo), including:

◼

identity search

◼

similarity search

◼

super/substructure search

◼

molecular formula search

As explained in a PubChem paper (https://bit.ly/2kirxky), whereas structure search can be performed in either an ‘asynchronous’ or ‘synchronous’ way, it is highly recommended to use the synchronous approach.
The synchronous searches are invoked by using the keywords prefixed with ‘fast’, such as fastidentity, fastsimilarity_2d, fastsimilarity_3d, fastsubstructure, fastsuperstructure, and fastformula.

Identity Search

Using PUG-REST

PUG-REST allows you to search the PubChem Compound database for molecules identical to the query molecule. PubChem’s identity search supports different contexts of chemical identity, which the user can specify using the optional parameter, “identity_type”. Here are some commonly-used chemical identity contexts.

◼

same_connectivity: returns compounds with the same atom connectivity as the query molecule, ignoring stereochemistry and isotope information.

◼

same_isotope: returns compounds with the same isotopes (as well as the same atom connectivity) as the query molecule. Stereochemistry will be ignored.

◼

same_stereo: returns compounds with the same stereochemistry (as well as the same atom connectivity) as the query molecule. Isotope information will be ignored.

◼

same_stereo_isotope: returns compounds with the same stereochemistry AND isotope information (as well as the same atom connectivity). This is the default.

The following code cell demonstrates how these different contexts of chemical sameness affects identity search in PubChem.

We begin by defining a function to perform a sameness comparison given one of these types and a SMILES string to search; the function is defined in a general way, but with default values of the arguments corresponding to an example query about the molecule “C(/C=C/Cl)Cl”:

In[]:=

samenessComparison[CIDType_String:"same_stereo_isotope",SMILES_String:"C(/C=C/Cl)Cl"]:=With[{url=URLBuild[{"https://pubchem.ncbi.nlm.nih.gov/rest/pug","compound/fastidentity/smiles/property/isomericsmiles/csv"},{"identity_type"CIDType}]},URLExecute[#,"CSV"]&@HTTPRequest[url,<|"Method""POST","Body"{"smiles"SMILES}|>]]

The function returns a list of lists of the CIDs and SMILES strings:

In[]:=

samenessComparison["same_stereo"]

Out[]=

{{CID,IsomericSMILES},{24726,C(/C=C/Cl)Cl},{102602172,[2H]/C(=C(/[2H])\Cl)/C([2H])([2H])Cl}}

It is often convenient to work with this in the form of a Dataset; the function repository provides a DatasetWithHeaders function that interprets the first row as a set of headers and constructs the relevant Dataset. Begin by retrieving the relevant function:

In[]:=

ResourceFunction["DatasetWithHeaders"]

Out[]=

[◼]	DatasetWithHeaders

Apply this function to our results:

In[]:=

[◼]	DatasetWithHeaders

@samenessComparison["same_stereo"]

Out[]=

CID	IsomericSMILES
24726	C(/C=C/Cl)Cl
102602172	[2H]/C(=C(/[2H])\Cl)/C([2H])([2H])Cl

Next, we want to append images of the molecules generated using the SMILES strings contained in the IsomericSMILES column. To do this, we define a function:

In[]:=

appendImage[data_Dataset]:=With[{graphicsFunction=MoleculePlot@Molecule[#["IsomericSMILES"]]&},data[All,Append[#,"image"graphicsFunction[#]]&]](*example*)appendImage@

[◼]	DatasetWithHeaders

@samenessComparison["same_stereo"]

Out[]=

CID	IsomericSMILES	image
24726	C(/C=C/Cl)Cl
102602172	[2H]/C(=C(/[2H])\Cl)/C([2H])([2H])Cl

Let’s take a look at the values for each of the possible query types by construction an Association from the query name to the Dataset of results generated above:

In[]:=

AssociationMapappendImage@

[◼]	DatasetWithHeaders

@samenessComparison[#]&,{"same_stereo_isotope","same_stereo","same_isotope","same_connectivity"}//Dataset

Out[]=

same_stereo_isotope

CID	IsomericSMILES	image
24726	C(/C=C/Cl)Cl

same_stereo

CID	IsomericSMILES	image
24726	C(/C=C/Cl)Cl
102602172	[2H]/C(=C(/[2H])\Cl)/C([2H])([2H])Cl

same_isotope

CID	IsomericSMILES	image
24726	C(/C=C/Cl)Cl
24883	C(C=CCl)Cl
5280970	C(/C=C\Cl)Cl

same_connectivity

CID	IsomericSMILES	image
24726	C(/C=C/Cl)Cl
24883	C(C=CCl)Cl
5280970	C(/C=C\Cl)Cl
102602172	[2H]/C(=C(/[2H])\Cl)/C([2H])([2H])Cl
131875718	[2H]C(=C([2H])Cl)C([2H])([2H])Cl

Using the PubChem Service Connection

As introduced in the previous exercises, the PubChem Service Connection provides a convenient wrapper for dealing with PubChem that hides many of the details of the PUG-REST API. Recall that we can request a compound ID by providing the SMILES strings:

In[]:=

cids=ServiceExecute["PubChem","CompoundCID",{"SMILES""C(/C=C/Cl)Cl"}]

Out[]=

CompoundID

{24726}

The documentation about “Requests” contains documentation on the various options. The type of CID returned is specified by the “CIDType” option. For example, to return the “same_stereo” examples above:

In[]:=

ServiceExecute["PubChem","CompoundCID",{"SMILES""C(/C=C/Cl)Cl","CIDType""SameStereo"}]

Out[]=

CompoundID

{24726,102602172}

Just like the manual PUG-REST calls, matches can be restricted to compounds with the same stereochemistry, isotopes or connectivity. The PUG-REST names are written in the typical Mathematica CamelCase string style, without underscores. The only exception is the PUG-REST “same_stereo_isotope” which is specified as “Original” in Mathematica (this reasonable, as “same_stereo_isotope” does in fact restrict it to the originally specified SMILES string, and is the default setting if no “CIDType” option is specified). Below is the parallel example to the PUG-REST example above (for brevity, we will not bother to draw the structures):

In[]:=

AssociationMap[ServiceExecute["PubChem","CompoundCID",{"SMILES""C(/C=C/Cl)Cl","CIDType"#}]&,{"Original","SameStereo","SameIsotopes","SameConnectivity"}]//Dataset

Out[]=

Original

CompoundID

{24726}

SameStereo

CompoundID

{24726,102602172}

SameIsotopes

CompoundID

{24726,5280970,24883}

SameConnectivity

CompoundID	24726
	131875718
	102602172
	5280970
	24883

Exercise:

Exercise 1a: Find compounds that have the same atom connectivity and isotope information as the query molecule:

In[]:=

query="CC1=CN=C(C(=C1OC)C)C[S@](=O)C2=NC3=C(N2)C=C(C=C3)OC";

For each compound returned from the search, retrieve the following information:

◼

CID

◼

Isomeric SMILES string

◼

chemical synonyms (for simplicity, print only the five synonyms that first occur in the name list retrieved for each compound)

◼

Structure image (you may either retrieve this from PUG-REST or use MoleculePlot to compute it from the Isomeric SMILES string.

In[]:=

(*Writeyourcodeinthiscell.*)

Similarity search

2D- and 3D- Similarity Searching using PUG-REST

PubChem supports 2-dimensional (2-D) and 3-dimensional (3-D) similarity searches. Because molecular similarity is not a measurable physical observable but a subjective concept, many approaches have been developed to evaluate it. Detailed discussion on how PubChem quantifies molecular similarity, read the following LibreTexts page: Searching PubChem Using a Non-Textual Query (https://bit.ly/2lPznCo)

The code cell below demonstrates how to perform 2-D and 3-D similarity searches:

In[]:=

prolog="https://pubchem.ncbi.nlm.nih.gov/rest/pug";myData=<|"smiles"->"C1COCC(=O)N1C2=CC=C(C=C2)N3C[C@@H](OC3=O)CNC(=O)C4=CC=C(S4)Cl"|>;url=URLBuild[{prolog,"/compound/fastsimilarity_2d/smiles/cids/txt"},{"Threshold"99}]cids=URLExecute[#,"CSV"]&@HTTPRequest[url,<|"Method""POST","Body"myData|>]Print["# Number of CIDs: ",Length[cids]]

Out[]=

https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/fastsimilarity_2d/smiles/cids/txt?Threshold=99

Out[]=

{{9875401},{6433119},{11524901},{68152323},{25255944},{25190310},{25164166},{145624246},{145624236},{144489887},{143433422},{123868009},{56598114},{56589668},{11994745},{25190129},{25190130},{25190186},{25190187},{25190188},{25190189},{25190190},{25190248},{25190249},{25190250},{25190251},{25190252},{25190311},{25255845},{25255945},{25255946},{49849874},{133687098}}

# Number of CIDs: 33

It is worth mentioning that the parameter name “Threshold” is case-sensitive. If “threshold” is used (rather than “Threshold”), it will be ignored and the default value (0.90) will be used for the parameter. As a matter of fact, all optional parameter names in PUG-REST are case-sensitive, as demonstrated below:

In[]:=

url=AssociationMap[(*constructURLsforbothcases*)URLBuild[{prolog,"/compound/fastsimilarity_2d/smiles/cids/txt"},{#95}]&,{"Threshold","threshold"}]Map[(*Constructtherequest,execute,andcomputethelengthofresultsforeachURL*)Length@URLExecute[#,"CSV"]&@HTTPRequest[#,<|"Method"->"POST","Body"myData|>]&,url]

Out[]=

Thresholdhttps://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/fastsimilarity_2d/smiles/cids/txt?Threshold=95,thresholdhttps://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/fastsimilarity_2d/smiles/cids/txt?threshold=95

Out[]=

Threshold227,threshold1054

It is possible to run 3-D similarity search using PUG-REST. However, because 3-D similarity search takes much longer than 2-D similarity search, it often exceeds the 30-second time limit and returns a time-out error, especially when the query molecule is big.

In addition, for 3-D similarity search, it is not possible to adjust the similarity threshold (that is, the optional “Threshold” parameter does not work). 3-D similarity search uses a shape-Tanimoto (ST) of >=0.80 and a color-Tanimoto (CT) of >=0.50 as a similarity threshold. Read the LibreTexts page for more details (https://bit.ly/2lPznCo).

In[]:=

myData=<|"smiles""CC(=O)OC1=CC=CC=C1C(=O)O"|>;url=URLBuild[{prolog,"compound/fastsimilarity_3d/smiles/cids/txt"}];Length@URLExecute[#,"CSV"]&@HTTPRequest[url,<|"Method"->"POST","Body"myData|>]

Out[]=

2496

2D- and 3D- Similarity Searching using the PubChem Service Connection

The PubChem Service Connection provides convenience function for searching 2D similarity searching. Repeating the query from above:

In[]:=

example="C1COCC(=O)N1C2=CC=C(C=C2)N3C[C@@H](OC3=O)CNC(=O)C4=CC=C(S4)Cl";ServiceExecute["PubChem","CompoundCID",{"SMILES"example,Method"Similarity2DSearch","Threshold"99}]

Out[]=

CompoundID	9875401	6433119	11524901	68152323	25255944	25190310	25164166	145624246	145624236	144489887	143433422
	123868009	56598114	56589668	11994745	25190129	25190130	25190186	25190187	25190188	25190189	25190190
	25190248	25190249	25190250	25190251	25190252	25190311	25255845	25255945	25255946	49849874	133687098

This returns the same number of compounds found above:

In[]:=

%["CompoundID"]//Length

Out[]=

Similarly, it supports 3D similarity searching (repeating the example above):

In[]:=

example="CC(=O)OC1=CC=CC=C1C(=O)O";result=ServiceExecute["PubChem","CompoundCID",{"SMILES"example,Method"Similarity3DSearch"}];result["CompoundID"]//Length

Out[]=

2496

Exercise:

Exercise 2a: Perform 2-D similarity search with the following query, using a threshold of 0.80 and find the macromolecule targets of the assays in which the returned compounds were tested. You will need to take these steps:

◼

Run 2-D similarity search using the SMILES string as a query (with Threshold=80).

◼

Retrieve the AIDs in which any of the returned CIDs was tested “active”.

◼

Retrieve the gene symbols of the targets for the returned AIDs.

In[]:=

(*Writeyourcodeinthiscell.*)(*examplequerytouse*)query="[C@@H]23C(=O)[C@H](N)C(C)[C@H](CCC1=COC=C1)[C@@]2(C)CCCC3(C)C";

Substructure/Superstructure search

When a chemical structure occurs as a part of a bigger chemical structure, the former is called a substructure and the latter is referred to as a superstructure (https://bit.ly/2lPznCo). PUG-REST supports both substructure and superstructure searches. For example, below is an example for substructure search using the core structure of antibiotic drugs called cephalosporins as a query (https://en.wikipedia.org/wiki/Cephalosporin). As usual, we construct a URL defining the REST call for the search, and perform a POST operation to retrieve the results:

In[]:=

query="C12(SCC(=C(N1C([C@H]2NC(=O)[*])=O)C(=O)O[H])[*])[H]";myData=<|"smiles"query|>;url=URLBuild[{prolog,"compound/fastsubstructure/smiles/cids/txt"},{"Stereo""exact"}]URLExecute[HTTPRequest[url,<|Method"POST","Body"myData|>],"CSV"]//Length(*NumberofCIDs*)

Out[]=

https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/fastsubstructure/smiles/cids/txt?Stereo=exact

Out[]=

25573

The same search can be performed using the PubChem Service Connection:

In[]:=

ServiceExecute["PubChem","CompoundCID",{"SMILES"query,Method"SubstructureSearch","Stereo""Exact"}]

Out[]=

CompoundID	27447	6328657	5742673	5479529	71309	51039
	38103	33255	6024	6915944	5484735	5479530
	5362065	5282242	456255	47965	44187	41375
	30699	19150	9570757	6533629	6335986	5284529
	91562	62921	43708	43594	43507	16212182
	9937686	5361202	636405	20848883	6398970	5479527
	5281006	135413542	57515946	11954009	9870843	6410757
	167441	65755	54362	51038	47896	87900910
	87060529	9571075	6410758	6399253	6321411	5487888
	5361871	5284527	456256	62977	47964	88046200
	87716431	56841980	53850837	52918380	23664040	11655595
	9916466	9894479	9690105	6914282	6604438	6436628
	5282241	194216	71736	65536	135536952	135413544
	129893484	90437953	57514685	54121176	50986970	45266506
	21124775	16219159	12971804	11592969	9951998	9915819
	9578573	9578062	9577223	6436140	6436055	6336505
	6336480	6321413	5491577	5491294	5464257	5361919
	5361848	5284528	166592	133358	123816	68940
	68666	65476	62979	50194	40958	40240
	135456161	134714754	71306765	66957706	56986002	53952828
	25573 total ›

It is important to remember that if the query structure is not specific enough (for example too small of a fragment), a substructure search will return more matches than the PubChem server can handle. For example, attempting a substructure search using “C-C” as a query, will return an error, because PubChem has ~96 million (organic) compounds with more than two carbon atoms and most of them will have the “C-C” unit. Therefore, if you get a “time-out” error while doing substructure search, consider providing a more specific structure as an input query.

Exercise

Exercise 3a: Below is the SMILES string for a HCV (Hepatitis C Virus) drug (Sovaldi). Perform substructure search using this SMILES string as a query, identify compounds that are mentioned in patent documents, and create a list of the patent documents that mentioning them.

◼

Use the default options for substructure search.

◼

Use the “XRefs” operation to retrieve Patent IDs associated with the returned compounds.

◼

For simplicity, ignore the CID-Patent ID mapping. (That is, no need to track which CID is associated with which patent document.)

In[]:=

(*Writeyourcodeinthiscell.*)(*examplequerytouse*)query="C[C@@H](C(=O)OC(C)C)N[P@](=O)(OC[C@@H]1[C@H]([C@@]([C@@H](O1)N2C=CC(=O)NC2=O)(C)F)O)OC3=CC=CC=C3";

Molecular formula search

Strictly speaking, molecular formula search is not structure search, but its PUG-REST request URL is constructed in a similar way to structure searches like identity, similarity, and substructure/superstructure searches:

In[]:=

query="C22H28FN3O6S";(*MolecularformulaforCrestor(Rosuvastatin:CID446157)*)url=URLBuild[{prolog,"compound/fastformula",query,"cids/txt"}]URLExecute[url,"CSV"]//Length(*NumberofCIDS*)

Out[]=

https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/fastformula/C22H28FN3O6S/cids/txt

Out[]=

197

It is possible to allow other elements to be present in addition to those specified by the query formula, by setting the AllowOtherElements option:

In[]:=

url=URLBuild[{prolog,"compound/fastformula",query,"cids/txt"},{"AllowOtherElements""true"}]URLExecute[url,"CSV"]//Length(*NumberofCIDS*)

Out[]=

https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/fastformula/C22H28FN3O6S/cids/txt?AllowOtherElements=true

Out[]=

220

Both types of queries can also be performed using Mathematica’s PubChem Service Connection, by specifying a “Formula” and setting the Method to “FormulaSearch”:

In[]:=

ServiceExecute["PubChem","CompoundCID",{"Formula"query,Method"FormulaSearch"}]

Out[]=

CompoundID	446157	67022109	6239562	6439133	656606	54639947
	54639946	54639945	54639944	54639943	54639942	54639941
	54639940	4979943	129806	136629448	118988812	118988808
	91591524	91571993	91492122	91433585	91425214	91345552
	91179466	90911029	90842570	90828679	90116753	90116386
	89610376	87833241	87707820	87666242	87658164	87634271
	87587801	87587799	87392903	87285484	70240861	69777636
	69576289	69496056	69253231	68954899	68727673	67495604
	67303862	67143835	67022111	66898519	66718458	58669051
	58254531	57952027	57952014	52934049	25013915	24739633
	23600972	16036317	12136801	11329129	11225410	11213815
	6436659	6321284	144873063	144199007	143305133	142756494
	142756493	142727968	142175996	141625068	141597421	137380749
	129850360	129850340	129850335	124128296	122444882	117911168
	102274675	91230150	91146810	91094560	90656916	88037905
	87932369	87393492	85332042	76673978	75964998	69648208
	69648207	69292144	69085665	69049845	68858340	68535826
	67422510	67143478	66955499	66848352	54644333	54644332
	54644331	54644330	54644329	54644328	54644327	54644326
	25235015	23499823	17843663	12136803	12114745	10288856
	197 total ›

The “AllowOtherElements” is specified using Mathematica’s Boolean value symbols (True, False), rather than the strings (“true”, “false”) used when manually constructing the PUG-REST call:

In[]:=

ServiceExecute["PubChem","CompoundCID",{"Formula"query,Method"FormulaSearch","AllowOtherElements"True}]

Out[]=

CompoundID	446157	54613184	67022109	6239562	6439133	656606	54639947	54639946	54639945	54639944
	54639943	54639942	54639941	54639940	4979943	129806	136629448	118988812	118988808	118540641
	118540388	118540387	91591524	91571993	91492122	91433585	91425214	91345552	91179466	90911029
	90842570	90828679	90242867	90116753	90116386	89610376	87833241	87707820	87666242	87658164
	87634271	87587801	87587799	87422318	87392903	87285827	87285484	87285483	70240861	69799036
	69798554	69777636	69576289	69496056	69253231	68954899	68727673	67495604	67303862	67143835
	67022111	66898519	66718458	66607916	58669051	58254531	57952027	57952014	52934049	25013915
	24758243	24739633	23600972	16036317	12136801	11329129	11225410	11213815	9892933	6436659
	6321284	144873063	144199007	143305133	142756494	142756493	142727968	142175996	141625068	141597421
	137380749	129850360	129850340	129850335	124128296	122444882	117911168	102274675	91230150	91146810
	91094560	90656916	88037905	87932369	87393492	85332042	76673978	75964998	69648208	69648207
	69292144	69085665	69049845	68858340	68535826	67527100	67422510	67143478	66955499	66848352
	54644333	54644332	54644331	54644330	54644329	54644328	54644327	54644326	25235015	23499823
	17843663	12136803	12114745	10288856	10174191	10162921	10185662	18508243	22578818	22578839
	22895449	24451120	24546295	24783389	24783390	24783391	24783646	25207817	25241189	25241190
	25241191	25241192	25241225	25241235	31724329	31724520	41620973	42066903	46653596	53248563
	53340670	55343944	55427363	55428467	55452025	55452271	55463219	55473535	55503400	55674294
	55719320	55849611	56194540	56194733	56416974	56417486	56493653	57347724	71752110	73442492
	74057344	76973131	76973950	89623416	94500895	100115815	100115823	100115840	100115848	100115863
	100115874	100626050	100626065	100626091	100626107	100626133	100626152	118985233	122164982	123910268
	220 total ›

Exercise

Exercise 4a: The general molecular formula for alcohols is CnH(2n+2)O [for example, CH4O (methanol), C2H6O (ethanol), C3H8O (propanol), etc]. Run molecular formula search using this general formula for n=1 through 20 and retrieve the XLogP values of the returned compounds for each value of n. Print the minimum and maximum XLogP values for each n value.

In[]:=

(*Writeyourcodeinthiscell.*)

Attributions

Adapted from the corresponding OLCC 2019 Python Assignment:
https://chem.libretexts.org/Courses/Intercollegiate_Courses/Cheminformatics_OLCC_ (2019)/4._Searching _Databases _for _Chemical _Information/4.6%3 A_Python _Assignments

Cite this as: Joshua Schrier, "04 Structure Search" from the Notebook Archive (2020), https://notebookarchive.org/2020-10-ebo2bs0