Downloading Datasets

Fetching datasets from the Internet

Let's Get Data

http://www.emboss.org

$ man wget

Downloading From The Web

Using wget to download PDB data-files

$ mkdir structures

$ cd structures

$ wget ftp://ftp.rcsb.org/pub/pdb/data/structures/all/pdb1m7t.ent.Z

$ wget ftp://ftp.rcsb.org/pub/pdb/data/structures/all/pdb1lqt.ent.Z

$ gzip -d pdb1m7t.ent.Z pdb1lqt.ent.Z

$ wget --mirror ftp://ftp.rcsb.org/pub/pdb/data/structures/all/pdb

Mirroring a dataset

Maxim 9.1

Only download a dataset when absolutely necessary. Consider the implications of doing so first

structures/ftp.rcsb.org/pub/pdb/data/structures/all/pdb

$ wget --output-file=log --mirror --http-user=anonymous

--http-passwd=email@where.ever.net

--directory-prefix=structures/mmCIF

--no-host-directories

--cut-dirs=6 ftp://ftp.rcsb.org/pub/pdb/data/structures/all/pdb

Smarter mirroring

$ wget ftp://beta.rcsb.org/pub/pdb/uniformity/data/mmCIF/all/1ger.cif.Z

Downloading a subset of a dataset

Where To From Here

2004-04-02T16:51:44