This page explains the requirements and how to run the Piwik Log Analytics tool to import your server logs in Piwik.
Once you have Piwik running, you will find the script in misc/log-analytics/import_logs.py
$ python /path/to/piwik/misc/log-analytics/import_logs.py
This will display the help information. The only required parameter is
to specify the Piwik base URL. Then, you can specify one or many log files to import.
There are many more options available. See the help output, and the README for more information and explanations about available parameters.
For example, if you wish to track all requests (static files, bot requests, http errors, http redirects) the following command would be used:
python /path/to/piwik/misc/log-analytics/import_logs.py --url=http://analytics.example.com access.log --idsite=1234 --recorders=4 --enable-http-errors --enable-http-redirects --enable-static --enable-bots
By default, the script does not track static files (JS, CSS, images, etc.) and excludes all bot traffic.
You can enable these using the following commands:
will track search/spam bots in Piwik, using a custom variable with the name of the bot. When enabled, the log file will take longer to process since all bot page views are sent to Piwik.
will specify tracking of all static files (images, JS, CSS) in Piwik. This will add some time to the general log file processing.
will specify tracking of HTTP errors (4xx, 5xx status) as a page view in Piwik, with a custom variable HTTP-code set to 404, 500, etc. The page title for this page view will show the URL referrer if it is specified in the log file (which can help finding out which pages have a link to a 404 for example).
will track HTTP redirect (302,301,3xx) as a page view, with a custom title, and a custom variable. Note: HTTP status 304 responses (“Not modified”) are tracked as page views.
will enable the reverse DNS (used to generate the Visitors > Providers report), expect a big performance hit as reverse DNS is very slow.
specifies the number of threads: we recommend setting it to the number of CPU cores in the system (or slightly more or less depending on your server configuration)
The importer uses the bulk tracking feature of Piwik to achieve greater speed. By default, 300 pageviews (or log lines) will be sent to Piwik at once. You can experiment with this number to try and achieve better performance, but there is an upper limit to the speed you can get.
There are several ways to exclude particular log lines or visitors from being tracked.