Stata Resources for Data Management

Attachment Size
Jan27_2014_material_wiki.pdf25.79 KB 25.79 KB
Programs versus do files What's the difference_v2.pdf610.67 KB 610.67 KB
Puetz gettoken guide.pdf266.52 KB 266.52 KB
Puetz closing comments guide.pdf55.6 KB 55.6 KB
traveltime3 geocode3_b.pdf180.25 KB 180.25 KB
spmap.pdf35.03 KB 35.03 KB
STATA matrix construction guide.pdf231.11 KB 231.11 KB
edgelists.pdf187.44 KB 187.44 KB
Community Resource 1.doc35 KB 35 KB
research-log-blank.doc42 KB 42 KB
wf2-data-registry.xls46.5 KB 46.5 KB
wf2-directory-design.xls20 KB 20 KB

This page is the collective product of my Stata Programming class from Spring 2014 (each segment in lines was a course day). We encourage other people to use and build upon these materials. I would appreciate links to material that builds on these topics, so I can integrate links into this page.

It is organized according to the original syllabus-- the table below contains the topics, readings, and course presentation files created for each course day. All readings should have hotlinks that worked at the time of the course (sorry if they break over time as people move their stuff around). Email me if something is broken and I can see if I have an archive copy.

As for the books, I used Scott Long's Workflow of Data Analysis Using Stata and Newton and Cox's Seventy-Six Stata Tips. Those are the only readings below without hotlinks.

 


UNIT 1: GENERAL INTRODUCTION AND CONVENTIONS


Replicability, Efficiency, and Elegance as Programming Goals
Naming Conventions for Files
Writing and Reading Documentation

Help menu and items: http://youtu.be/UpXNMeTzmuI
PDF Documentation: http://youtu.be/KPHxC-HyrMk
Long, Ch 1: "Introduction"
Long,  Ch 2: "Planning, organizing and documenting"

Learning resource produced in class: http://prezi.com/kwkcljweozhp/?utm_campaign=share&utm_medium=copy&rc=ex0share & http://prezi.com/mh6pasy33t7t/?utm_campaign=share&utm_medium=copy&rc=ex0share

Also: A cheat sheet and template files to help with planning, organizing, and documenting: http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/Community%20Resource%201.doc http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/research-log-blank.doc http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/wf2-data-registry.xls http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/wf2-directory-design.xls


An introduction to do files
Commenting
Debugging errors Stata finds
Finding errors Stata doesn’t notice

Long,  Ch 3: “Writing and Debugging Do Files”
Long, Ch 5.2-5.4 from “Names, Notes and Labels”

If you are still confused:
http://www.ssc.wisc.edu/sscc/pubs/sfr-dofiles.htm

Learning resource produced in class:
http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/do_file_CR.pdf
http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/Jan27_2014_material_wiki.pdf


Ado Files and the SSC Archives + Getting Help

http://www.ssc.wisc.edu/sscc/pubs/sfr-dofiles.htm

Stata tip 30: “May the Source be With You (viewsource)”
Long, Appendix: “How Stata Works”
General overview for Stata beginners and relative novices: http://www.wiwi.uni-muenster.de/ioeb/Downloads/Forschen/Pfaff/Introduct…

Learning resource produced in class: http://youtu.be/iP-8ne6j9ZI
PDF Guide from https://statacommunityresources.wordpress.com


UNIT 2: STATA BASICS


Data Storage types
Variable Naming
Labels and Notes

Variable Types and Precision and String heading: http://www.ssc.wisc.edu/sscc/pubs/sfr-data.htm
Long, Ch 5.5-5.10: "Names, notes, and labels"
Drop, Keep, Rename, Labels: http://www.ssc.wisc.edu/sscc/pubs/sfr-data.htm
Describe and Labels: http://www.ats.ucla.edu/stat/stata/modules/labels.htm
Importing Data: http://www.stata.com/support/faqs/data-management/converting-excel-file… (from a common format like Excel)
Stata Tip 35: Detecting Whether Data Have Changed (Datasignature)
Other ways to get data into Stata: Pages 3-11 from http://fmwww.bc.edu/GStat/docs/StataInter.pdf"
http://www.dawnteele.com/dt.11/technology_files/The%20Stata%20Bible%202…

Learning resource produced in class: http://dsesate.tumblr.com
http://soctips.tumblr.com/


Importing and summarizing data

Long, Ch 6.1-6.2 & 6.4-6.5: “Cleaning your data”
Stata tip 66: “ds- a hidden gem

Learning resource produced in class: http://snardinstata.tumblr.com/
http://meganmckendry.wix.com/import-and-sum-data


Basic Data Manipulation

Long, Ch 6.3: “Cleaning your data”
Stata Tip 52: Generating composite categorical variables
Stata Tip 2: Building with floors and ceilings
help egen in Stata—mostly focus on the functions available
http://www.ats.ucla.edu/stat/stata/faq/cut.htm

If the Long was confusing to you, also read:
replace and recode:
http://www.ats.ucla.edu/stat/stata/modules/vars.htm
Replace and indicator variables: http://www.ssc.wisc.edu/sscc/pubs/sfr-data.htm (through Indicator variables)
Using tabulate to recode into dummies: “Answer 2 of 3: Use tabulate” http://www.stata.com/support/faqs/data-management/creating-dummy-variab… & http://kb.iu.edu/data/bajq.html

If egen was confusing to you, also read:
http://www.ssc.wisc.edu/sscc/pubs/sfr-data.htm
http://stataproject.blogspot.com/2007/12/step-4-thank-god-for-egen-comm…

Learning resource produced in class: http://karinagsalazar.edublogs.org/2014/02/16/basicdatamanipulation/


UNIT 3: INTERMEDIATE DATA MANIPULATION


By, System Variables, Return Codes, Egenmore

By from Stata PDF Documentation (see course webpage)
System variables: http://www.stata.com/help.cgi?_variables
Using system variables: http://www.cpc.unc.edu/research/tools/data_analysis/statatutorial/misc/…
Return codes and values:
Long 4.2: “Information returned by Stata commands http://www.ats.ucla.edu/stat/stata/faq/returned_results.htm
Help egenmore (from Stata)
Stata tip 14: Using value labels in expressions

Learning resource produced in class:
http://funwithstata.blogspot.com/
http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/By%2C%20System%20Variables%2C%20Return%20Codes%2C%20Egenmore%2C%20Year%202%20CR.pdf


Regular Expressions

Regular expressions available in Stata: http://www.stata.com/support/faqs/data-management/regular-expressions/
Regular expression commands in Stata: http://repec.org/wcsug2007/medeiros_reg_ex.pdf
Example of using a regular expression: http://www.ats.ucla.edu/stat/stata/faq/regex.htm
Stata Tip 60: Making fast and easy changes to files with filefilter
help filefilter (when in Stata)

Learning resource produced in class:
http://soc596.blogspot.com/
Regular Expressions, year 2.pdf


Manipulating Entire Datasets: Append, Merge, Collapse

Stata Tip 5: Ensuring programs preserve data sort order
help append (in Stata)
Append: http://www.cpc.unc.edu/research/tools/data_analysis/statatutorial/examp… (including the Q&A below it)
Stata Tip 73: Append with care!
Long, Ch 6.6: Cleaning your data
help collapse (in Stata)
Collapse: http://www.ats.ucla.edu/stat/stata/modules/collapse.htm
Applying merge to data cleaning: Stata Tip 64: Cleaning up user-entered string variables

If Long was unclear to you:
Merge overview: http://www.cpc.unc.edu/research/tools/data_analysis/statathttp://jearl…
1-to-1 merging: http://www.cpc.unc.edu/research/tools/data_analysis/statatutorial/examp… & http://www.cpc.unc.edu/research/tools/data_analysis/statatutorial/examp…
One to many merging: http://www.cpc.unc.edu/research/tools/data_analysis/statatutorial/examp…
Merge and append: http://dss.princeton.edu/training/Merge101.pdf

Learning resource produced in class:
Append: https://www.youtube.com/watch?v=C7nAW4Lo01U
Merge 1:1, m:1: https://www.youtube.com/watch?v=2Fqz4ARUaFc
Merging multiple datasets in a row: https://www.youtube.com/watch?v=yVFqz-B9WQA
Collapse: https://www.youtube.com/watch?v=Y-rcs1ogLp8
Manipulating%20Entire%20Datasets%2C%20Append%2C%20Merge%2C%20Collapse%2C%20Year%202.pdf


Reshaping Datasets and Transferring and Preserving Data

Reshape in general: http://www.ats.ucla.edu/stat/stata/notes/reshape.htm
Reshape to long: http://www.ats.ucla.edu/stat/stata/modules/reshapel.htm
Reshape to wide: http://www.ats.ucla.edu/stat/stata/modules/reshapew.htm
Stata Tip 45: Getting those data into shape (reshape)
StatTransfer manual, Pp 50-78
Long, Ch 8: "Protecting your files"

Learning resource produced in class:
On Reshape: https://www.youtube.com/watch?v=CMey0Qu08xo
On StatTransfer: https://www.youtube.com/watch?v=a3g1cjWhcIg Note: you could run the commands he shows you in the video by saving the commands into a text file with a .stcmd extension, placing the file in the directory that contains the things you want to transfer, and then double-clicking on the program in Windows-- it will work the same--but easier for you--as running it through the command window)
On Backups and Mirrors/Preserving Data: http://statsrock.files.wordpress.com/2014/03/backup1.pdf but I think the whole site is worth looking at b/c it has one of the funniest tag lines ever (see http://statsrock.files.wordpress.com)
Reshaping Datasets, year 2.pdf


UNIT 4: LOOPS AND PROGRAMS


Intro to loops

Long, Ch 4 (pp. 83-105, except 4.3.3): "Automating your work"
Macros: "Macros" on http://www.ssc.wisc.edu/sscc/pubs/stata_prog1.htm -- up to, but not including, “Nested Loops”
"B] macros" on http://pierrefrancois.wifeo.com/documents/Intro-Stata---LSE-III.pdf (pp 10-14)
Tempfiles: http://www.stata.com/statalist/archive/2004-01/msg00542.html

Learning resource produced in class:
https://www.youtube.com/watch?v=WsqvGdqzknY
http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/Intro%20to%20loops%2C%20Year%202.pdf


forval versus foreach, forval in detail, i & local j, nested loops

Long, Ch 4 (4.3.3 & 105-end of chapter): "Automating your work"
Forval: "For Loops & Nested Loops " on http://www.ssc.wisc.edu/sscc/pubs/stata_prog1.htm
http://www.ats.ucla.edu/stat/stata/modules/acrossvars.htm

Learning resource produced in class:
http://hrstenson.tumblr.com/ & the accompanying dataset  & do file
forval loops: https://youtu.be/KsUk3E93B88
foreach loops: https://youtu.be/tWvjjkOt5Kc
nested loops: https://youtu.be/0hZzSi0wrgU


loops continued, preventing errors (assert), and debugging loops (trace, pause, and capture)

http://www.stata-journal.com/sjpdf.html?articlenum=pr0005
Trace: http://www.stata.com/support/faqs/programming/debugging-program/
Stata Tip 32: Do Not Stop (do nostop)
Stata Tip 41: Monitoring loop iterations
help assert in Stata
help pause in Stata
help capture in Stata

Learning resource produced in class:
http://loopscontinued.tumblr.com/
Loops Continued, preventing and debugging errors, year 2.pdf


Post

[P] postfile
http://datatoday.blogspot.com/2011/11/using-postfile-in-stata.html
http://www.michaelnormanmitchell.com/stow/posting-results-regression.ht…
Stata tip 54: Post your results (postfile and postclose)
http://www.stata.com/support/faqs/programming/post-command/

Learning resource produced in class:
http://stataresources.blogspot.com/2014/03/introduction-to-p-postfile-i…
Post, Year 2.pdf


Programs versus do files: what’s the difference? And, passing arguments

Readings:
http://www.ssc.wisc.edu/sscc/pubs/stata_prog_old.htm “Programs” section
http://www.cpc.unc.edu/research/tools/data_analysis/statatutorial/labor… (read entire page, including questions and answers at the bottom of the page)
http://www.stata.com/manuals13/u18.pdf , sections 18.4, 18.4.1

Learning resource produced in class: http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/…
Writing Programs in Stata


Gettoken and more on arguments

http://www.stata.com/manuals13/u18.pdf , section 18.4.6

Learning resource produced in class: http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/…


Closing Thoughts on Stata

Learning resource produced in class: http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/…


UNIT 5: INTERFACING WITH OTHER PROGRAMS


Windows batch files (which have very similar, but often more powerful analogs in Unix, and hence Macs)

http://commandwindows.com/batch.htm
http://www.computerhope.com/batch.htm
http://ss64.com/nt/ (skim, but read: dir, cd, copy, del)
 

Learning resource produced in class: http://akikota.edublogs.org/2014/04/07/soc596a-windows-batch-file-mac-b…
Learning Resource for bash commands on unix machines:https://www.youtube.com/watch?v=OBdoshul8KU
Learning Resource for windows batch commands:https://www.youtube.com/watch?v=H3IS-xOMc9U


ODDS AND ENDS


Dates:

https://www.stanford.edu/group/ssds/cgi-bin/drupal/files/Guides/Working%20with%20Dates%20and%20Times%20in%20Stata.pdf


Working with Geo-Data:

http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/…
http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/…


Matrices in Stata (Intro):

http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/…


Transforming Edge Lists from Pajek:

http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/…
https://drive.google.com/file/d/0B7FJB3njFKSYUmh6bzZjR2VNOVU/
https://drive.google.com/file/d/0B7FJB3njFKSYeXRCcGhLbUNJakE/