Stata Resources for Data Management

This page is the collective product of my Stata programming classes (each segment below is roughly a course day in the class). We encourage other people to use and build upon these materials. I would appreciate links to material that builds on these topics, so I can integrate links into this page.

All readings should have hotlinks that worked at the time of the course (sorry if they break over time as people move their stuff around). Email me if something is broken and I can see if I have an archive copy.

As for the books, I used Scott Long's Workflow of Data Analysis Using Stata and Newton and Cox's Seventy-Six Stata Tips. Those are the only readings below without hotlinks.

 


UNIT 1: GENERAL INTRODUCTION AND CONVENTIONS


Replicability, Efficiency, and Elegance as Programming Goals
Naming Conventions for Files
Writing and Reading Documentation

Help menu and items: http://youtu.be/UpXNMeTzmuI
PDF Documentation: http://youtu.be/KPHxC-HyrMk
Long, Ch 1: "Introduction"
Long,  Ch 2: "Planning, organizing and documenting"

Learning resource produced in class: Explainer 1
Explainer 2

Also: Cheat sheets to help with planning, organizing, and documenting:
Cheat sheet 1
Cheat sheet 2
Cheat sheet 3
Cheat sheet 4
Worksheet to help you think about file organization


An introduction to do files
Commenting
Debugging errors Stata finds
Finding errors Stata doesn’t notice

Long,  Ch 3: “Writing and Debugging Do Files”
Long, Ch 5.2-5.4 from “Names, Notes and Labels”

If you are still confused:
http://www.ssc.wisc.edu/sscc/pubs/sfr-dofiles.htm

Learning resource produced in class:
Explainer 1
Explainer 2


Ado Files and the SSC Archives + Getting Help

http://www.ssc.wisc.edu/sscc/pubs/sfr-dofiles.htm

Stata tip 30: “May the Source be With You (viewsource)”
Long, Appendix: “How Stata Works”
General overview for Stata beginners and relative novices: https://www.scribd.com/document/339281306/Introduction-to-Stata-with-50…

Learning resource produced in class: Video Explainer 1
PDF Guide from https://statacommunityresources.wordpress.com


UNIT 2: STATA BASICS


Data Storage types
Variable Naming
Labels and Notes

Variable Types and Precision and String heading: http://www.ssc.wisc.edu/sscc/pubs/sfr-data.htm
Long, Ch 5.5-5.10: "Names, notes, and labels"
Drop, Keep, Rename, Labels: http://www.ssc.wisc.edu/sscc/pubs/sfr-data.htm
Describe and Labels: https://stats.idre.ucla.edu/stata/modules/labeling-data/
Importing Data: http://www.stata.com/support/faqs/data-management/converting-excel-file… (from a common format like Excel)
Stata Tip 35: Detecting Whether Data Have Changed (Datasignature)
Other ways to get data into Stata: Pages 3-11 from http://fmwww.bc.edu/GStat/docs/StataInter.pdf Omnibus Stata start-up guide: http://dawnteele.weebly.com/uploads/2/4/9/3/24931233/the_stata_bible_2…

Learning resource produced in class: http://dsesate.tumblr.com
http://soctips.tumblr.com/
Explainer 1


Importing and summarizing data

Long, Ch 6.1-6.2 & 6.4-6.5: “Cleaning your data”
Stata tip 66: “ds- a hidden gem

Learning resource produced in class: http://snardinstata.tumblr.com/
http://meganmckendry.wix.com/import-and-sum-data
Additional Explainer


Basic Data Manipulation

Long, Ch 6.3: “Cleaning your data”
Stata Tip 52: Generating composite categorical variables
Stata Tip 2: Building with floors and ceilings
help egen in Stata—mostly focus on the functions available
http://www.ats.ucla.edu/stat/stata/faq/cut.htm

If the Long was confusing to you, also read:
replace and recode:
http://www.ats.ucla.edu/stat/stata/modules/vars.htm
Replace and indicator variables: http://www.ssc.wisc.edu/sscc/pubs/sfr-data.htm (through Indicator variables)
Using tabulate to recode into dummies: “Answer 2 of 3: Use tabulate” http://www.stata.com/support/faqs/data-management/creating-dummy-variab… & http://kb.iu.edu/data/bajq.html

If egen was confusing to you, also read:
http://www.ssc.wisc.edu/sscc/pubs/sfr-data.htm
http://stataproject.blogspot.com/2007/12/step-4-thank-god-for-egen-comm…

Learning resource produced in class: http://karinagsalazar.edublogs.org/2014/02/16/basicdatamanipulation/
Additional Explainer


UNIT 3: INTERMEDIATE DATA MANIPULATION


By, System Variables, Return Codes, Egenmore

By from Stata PDF Documentation (see course webpage)
System variables: http://www.stata.com/help.cgi?_variables
Using system variables: http://www.cpc.unc.edu/research/tools/data_analysis/statatutorial/misc/…
Return codes and values:
Long 4.2: “Information returned by Stata commands http://www.ats.ucla.edu/stat/stata/faq/returned_results.htm
Help egenmore (from Stata)
Stata tip 14: Using value labels in expressions

Learning resource produced in class:
http://funwithstata.blogspot.com/
Explainer 2


Regular Expressions

Regular expressions available in Stata: http://www.stata.com/support/faqs/data-management/regular-expressions/
Regular expression commands in Stata: http://repec.org/wcsug2007/medeiros_reg_ex.pdf
Example of using a regular expression: http://www.ats.ucla.edu/stat/stata/faq/regex.htm
Stata Tip 60: Making fast and easy changes to files with filefilter
help filefilter (when in Stata)

Learning resource produced in class:
http://soc596.blogspot.com/
Regular Expressions, year 2.pdf
Exercises using tweets as text


Manipulating Entire Datasets: Append, Merge, Collapse

Stata Tip 5: Ensuring programs preserve data sort order
help append (in Stata)
Append: http://www.cpc.unc.edu/research/tools/data_analysis/statatutorial/examp… (including the Q&A below it)
Stata Tip 73: Append with care!
Long, Ch 6.6: Cleaning your data
help collapse (in Stata)
Collapse: http://www.ats.ucla.edu/stat/stata/modules/collapse.htm
Applying merge to data cleaning: Stata Tip 64: Cleaning up user-entered string variables

If Long was unclear to you:
Merge overview: http://www.cpc.unc.edu/research/tools/data_analysis/statathttp://jearl…
1-to-1 merging: http://www.cpc.unc.edu/research/tools/data_analysis/statatutorial/examp… & http://www.cpc.unc.edu/research/tools/data_analysis/statatutorial/examp…
One to many merging: http://www.cpc.unc.edu/research/tools/data_analysis/statatutorial/examp…
Merge and append: http://dss.princeton.edu/training/Merge101.pdf

Learning resource produced in class:
Append: https://www.youtube.com/watch?v=C7nAW4Lo01U
Merge 1:1, m:1: https://www.youtube.com/watch?v=2Fqz4ARUaFc
Merging multiple datasets in a row: https://www.youtube.com/watch?v=yVFqz-B9WQA
Collapse: https://www.youtube.com/watch?v=Y-rcs1ogLp8
Explainer 1
Explainer 2


Reshaping Datasets and Transferring and Preserving Data

Reshape in general: http://www.ats.ucla.edu/stat/stata/notes/reshape.htm
Reshape to long: http://www.ats.ucla.edu/stat/stata/modules/reshapel.htm
Reshape to wide: http://www.ats.ucla.edu/stat/stata/modules/reshapew.htm
Stata Tip 45: Getting those data into shape (reshape)
StatTransfer manual, Pp 50-78
Long, Ch 8: "Protecting your files"

Learning resource produced in class:
On Reshape: https://www.youtube.com/watch?v=CMey0Qu08xo
On StatTransfer: https://www.youtube.com/watch?v=a3g1cjWhcIg Note: you could run the commands he shows you in the video by saving the commands into a text file with a .stcmd extension, placing the file in the directory that contains the things you want to transfer, and then double-clicking on the program in Windows-- it will work the same--but easier for you--as running it through the command window)
On Backups and Mirrors/Preserving Data: http://statsrock.files.wordpress.com/2014/03/backup1.pdf but I think the whole site is worth looking at b/c it has one of the funniest tag lines ever (see http://statsrock.files.wordpress.com)
Reshaping Datasets, year 2.pdf
Reshape Exercises and Solutions


UNIT 4: LOOPS AND PROGRAMS


Intro to loops

Long, Ch 4 (pp. 83-105, except 4.3.3): "Automating your work"
Macros: "Macros" on http://www.ssc.wisc.edu/sscc/pubs/stata_prog1.htm -- up to, but not including, “Nested Loops”
"B] macros" on http://pierrefrancois.wifeo.com/documents/Intro-Stata---LSE-III.pdf (pp 10-14)
Tempfiles: http://www.stata.com/statalist/archive/2004-01/msg00542.html

Learning resource produced in class:
https://www.youtube.com/watch?v=WsqvGdqzknY
Explainer 1
Cheat sheet


forval versus foreach, forval in detail, i & local j, nested loops

Long, Ch 4 (4.3.3 & 105-end of chapter): "Automating your work"
Forval: "For Loops & Nested Loops " on http://www.ssc.wisc.edu/sscc/pubs/stata_prog1.htm
http://www.ats.ucla.edu/stat/stata/modules/acrossvars.htm

Learning resource produced in class:
http://hrstenson.tumblr.com/ & the accompanying dataset  & do file
forval loops: https://youtu.be/KsUk3E93B88
foreach loops: https://youtu.be/tWvjjkOt5Kc
nested loops: https://youtu.be/0hZzSi0wrgU
Explainer with exercises


loops continued, preventing errors (assert), and debugging loops (trace, pause, and capture)

http://www.stata-journal.com/sjpdf.html?articlenum=pr0005
Trace: http://www.stata.com/support/faqs/programming/debugging-program/
Stata Tip 32: Do Not Stop (do nostop)
Stata Tip 41: Monitoring loop iterations
help assert in Stata
help pause in Stata
help capture in Stata

Learning resource produced in class:
http://loopscontinued.tumblr.com/
Loops Continued, preventing and debugging errors, year 2.pdf
Debugging exercises


Post

postfile
http://datatoday.blogspot.com/2011/11/using-postfile-in-stata.html
http://www.michaelnormanmitchell.com/stow/posting-results-regression.ht…
Stata tip 54: Post your results (postfile and postclose)
http://www.stata.com/support/faqs/programming/post-command/

Learning resource produced in class:
http://stataresources.blogspot.com/2014/03/introduction-to-p-postfile-in-stata.html

Explainer 1
Explainer 2 with exercises


Programs versus do files: what’s the difference? And, passing arguments

Readings:
http://www.ssc.wisc.edu/sscc/pubs/stata_prog_old.htm “Programs” section
http://www.cpc.unc.edu/research/tools/data_analysis/statatutorial/labor… (read entire page, including questions and answers at the bottom of the page)
http://www.stata.com/manuals13/u18.pdf , sections 18.4, 18.4.1

Learning resource produced in class: Explainer 1
Writing Programs in Stata
Exercises


Gettoken and more on arguments

http://www.stata.com/manuals13/u18.pdf , section 18.4.6

Learning resource produced in class:
Explainer 1
Exercises


Closing Thoughts on Stata

Learning resource produced in class: Wrap-up resources


UNIT 5: INTERFACING WITH OTHER PROGRAMS


Windows batch files (which have very similar, but often more powerful analogs in Unix, and hence Macs)

http://commandwindows.com/batch.htm
http://www.computerhope.com/batch.htm
http://ss64.com/nt/ (skim, but read: dir, cd, copy, del)
 

Learning resource produced in class: Explainer 1
Video on bash commands for unix
Windows batch command video


ODDS AND ENDS


Dates:

https://www.stanford.edu/group/ssds/cgi-bin/drupal/files/Guides/Working%20with%20Dates%20and%20Times%20in%20Stata.pdf


Working with Geo-Data:

http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/…
http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/…


Matrices in Stata (Intro):

http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/…


Transforming Edge Lists from Pajek:

http://jearl.faculty.arizona.edu/sites/jearl.faculty.arizona.edu/files/…
https://drive.google.com/file/d/0B7FJB3njFKSYUmh6bzZjR2VNOVU/
https://drive.google.com/file/d/0B7FJB3njFKSYeXRCcGhLbUNJakE/