When you track a lot of data in your Piwik server, you may find that the calls to the Tracking API (piwik.php) become a scalability bottleneck. We have created a new plugin QueuedTracking that we release away for free(dom) to you, the community. This plugin will help with scaling your Piwik server to track dozens or hundreds of millions of requests per month.
This plugin writes all tracking requests into a Redis instance instead of directly into the database. This is useful if you have too many requests per second and your server cannot handle all of them directly (eg too many connections in nginx or MySQL). It is also useful if you experience peaks sometimes. Those peaks can be handled much better by using this queue. Writing a tracking request into the queue is very fast (a tracking request takes in total a few milliseconds) compared to a regular tracking request (that takes multiple hundreds of milliseconds). The queue makes sure to process the tracking requests whenever possible even if it takes a while to process all requests after there was a peak.
To get started, install the QueuedTracking plugin from the Marketplace. Click on the FAQ tab to learn more about the requirements and how to use the plugin on your Production Piwik server. For any bug report or feature request, please post them in the plugin’s issue tracker.
Contact Piwik Consultants if you need help with high traffic Piwik setup, or if you’d like experts to take care of your Piwik setup, configuration, maintenance and security.