Link

One neat trick to run better experiments

This Chrome Extension automatically flags potential data quality issues on supported experimentation platforms.

Install the Chrome Extension Manually check for SRM


What is Sample Ratio Mismatch? Why should we care?!

A Sample Ratio Mismatch (SRM) test can be used to detect a wide variety of data quality issues (paper) (presentation) that may affect online experiments (aka A/B tests). Only expected proportions and observed sample counts are required as input for this procedure, so this test can be used even in cases where experimenters only have access to summary statistics; such as when using third-party tools.

For more information about Sample Ratio Mismatch please read the Frequently Asked Questions.


About the project

Sample Ratio Mismatch Checker was created by Lukas Vermeer, Heinrich Mahr and Georgi Georgiev.

Supported platforms

  • Convert.com
  • Google Optimize
  • Omniconvert
  • SiteGainer/Symplify
  • Visual Website Optimizer (VWO)
  • Zoho PageSense

Known issues:

  • Convert: Auto-stopping variants triggers false SRM alert (#29)

License

Sample Ratio Mismatch Checker is distributed by an MIT license.

Contributing

You want to contribute to this project? That’s amazing! We’d love that! <3

Feel free to fork the repository on GitHub, make the changes you want to see, and issues a pull request. If you are not sure what to work on, we maintain a list of outstanding issues on the GitHub project. (If you don’t want to code, adding to that list of issues is also a great way to contribute.)

Thank you for making this project more awesome.

Chrome Web Store Chrome Web Store Chrome Web Store GitHub last commit