Data Analysis: Lessons Learned
Even as someone who analyzes data for a living, I often feel like I’m drowning in information trying to figure out how to make it useful. For organizations with limited capacity already working around the clock towards their mission, data analysis can seem unmanageable.
Over the last six months, we’ve supported ten organizations in the Social Impact Collaborative through the process of developing their data collection plans and analyzing data to inform programmatic improvement. No matter what kind of data you have and what analyses you are planning, there are three key steps – and three key lessons – to keep in mind when doing data analysis.
Step 1: Data Preparation
Data preparation involves cleaning and organizing your data so that it can be analyzed effectively and efficiently.
Lesson #1: Organizing your data is the most important part of your analysis. It can be easy to get overwhelmed by the sheer amount of data on your program. The key is ensuring that the data you analyze is aligned to what matters most. Here are some tips for organizing:
- Revisit your outcomes and indicators and identify the data that aligns to the questions you’re most interested in answering. We recommend creating a document that shows how your data align to the key outcomes/indicators. This type of dashboard can also be used to report results.
- Clean up any unnecessary fields in the data so you have a new data file that only includes the questions you plan on analyzing. However, we recommend always saving the original data file and creating a copy, so you don’t lose any information.
- Label all of your variables to align with the key outcomes and indicator language you’ve used in your planning. We recommend creating a codebook that describes the content, structure, and layout of the data you’ve collected.
See our Quantitative Analysis Guide for a sample codebook.
Step 2: Describing Your Data
With quantitative data, you’ll want to describe the number of participants represented in the data and the distribution of responses or behaviors. With qualitative data, you’ll want to summarize the major questions you asked and the dominant themes aligned to the key questions and outcomes you’re interested in.
Lesson #2: Knowing what type of data you have is critical. You don’t have to be a statistician to do meaningful data analysis. But you do need to know the type of data you have and the appropriate tests aligned to that type of data. There are three types of data. Continuous or numeric data captures information on a scale. Categorical or nominal data captures data in categories. Once you determine the type of data you have, you can choose the appropriate test.
- Continuous data (i.e., numeric data captured on a scale) is analyzed through measures of central tendency – i.e., mean, median, and mode. You can also test differences over time or between groups on certain continuous outcomes. For example, maybe you’re interested in comparing how member satisfaction changes from year to year.
- Categorical data (i.e., data organized in groups) is analyzed through frequencies or tabulations. You can also test differences over time or between groups on certain categorical outcomes. For example, maybe you’re interested in looking at the percentage of males vs. females who are members vs. non-members of your organization.
- Qualitative data (i.e., data from focus groups, interviews, or observations) is analyzed through coding of key themes. You can also code for differences across participants or sites. For example, maybe you’re interested in learning about the factors driving member satisfaction.
Step 3: Digging Deeper
Once you’ve analyzed your data descriptively, you should ask deeper questions about relationships or differences between groups in your data and think about what implications these results have for your work.
Lesson #3: Data is only useful if you use it. Once basic data has been collected, you should think about the trends and patterns that you see in the data and then define key priorities to explore further. We recommend engaging in a reflective cycle of inquiry that turns data into action by:
- Identifying priority challenges. Priorities should be aligned with current programmatic and organizational strategies and with your capacities as an organization.
- Exploring the root causes. Once you’ve identified your priority, it is important to identify the root causes of the current “state” of that priority. This includes exploring related problems/issues that may be impacting the priority. The more specific you can get, the better.
- Developing and implementing specific action plans. Your action plan should include how you will address the priority issue and bring about the results you wish to see. Your action plan should focus on one to two strategies at a time that are aligned with your staffing, program implementation plans, and funding constraints.
The way to achieve social impact goes beyond collecting and reporting on data; it involves asking the right questions and focusing in specific ways to use data to inform programmatic improvement.