Over the last few years I have been exposed to two different approaches to identifying and evaluating complex causal configurations within sets of data describing the attributes of projects and their outcomes. One is Qualitative Comparative Analysis (QCA) and the other is Predictive Analytics (and particularly Decision Tree algorithms). Both can work with binary data, which is easier to access than numerical data, but both require specialist software - which requires time and effort to learn how to use
In the last year I have spent some time and money, in association with a software company called Aptivate (Mark Skipper in particular) developing an Excel based package which will do many of the things that both of the above software packages can do, as well as provide some additional capacities that neither have.
This is called EvalC3, and is now available [free] to people who are interested to test it out, either using their own data and/or some example data sets that are available. The "manual" on how to use EvalC3 is a supporting website of the same name, found here: https://evalc3.net/ There is also a short introductory video here.
Its purpose is to enable users: (a) to identify sets of project & context attributes which are good predictors of the achievement of an outcome of interest, (b) to compare and evaluate the performance of these predictive models, and (c) to identify relevant cases for follow-up within-case investigations to uncover any causal mechanisms at work.
The overall approach is based on the view that “association is a necessary but insufficient basis for a strong claim about causation, which is a more useful perspective than simply saying “correlation does not equal causation”.While the process involves systematic quantitative cross-case comparisons, its use should be informed by within-case knowledge at both the pre-analysis planning and post-analysis interpretation stages.
The EvalC3 tools are organised in a work flow as shown below:
The selling points:
- EvalC3 is free, and distributed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
- It uses Excel, which many people already have and know how to use
- It uses binary data. Numerical data can be converted to binary but not the other way
- It combines manual hypothesis testing with algorithm based (i.e. automated) searches for good performing predictive models
- There are four different algorithms that can be used
- Prediction models can be saved and compared
- There are case-selection strategies for follow-up case-comparisons to identify any casual mechanisms at work "underneath" the prediction models
If you would like to try using EvalC3 email rick.davies at gmail.com
Skype video support can be provided in some instances. i.e. if your application is of interest to me :-)
