7 Reasons Why Your Commercial Site NEEDS Piwik…

Guest Blogger Marc Lindsay Is An Internet Marketer & Runs An SEO Company – mercurianmedia.com.au.

This is the first post of a new series all about “Using Piwik Effectively” and in a live environment.  Designed to help both businessman & hobbyist make the most out of Piwik.

Analytics Set Free - No More Constraints

There is no doubt that we are now in a world where digital data, bits & bytes, 1′s & 0′s all have a value, and a very significant value at that.  Just look at Google as a classical example of how valuable the digital world has now become.

Without data Google would have nothing to work with and I would be out of a job.

Previously when using say radio advertising or TV slots it has been a very hit and miss sort of game.  You don’t have an in-depth view into EXACTLY what it is that your viewer is looking for, so you would have to profile a demographic and then hope your offer or advert was appealing enough to spark interest.

Now bring it into today’s digital world and you can track almost everything.

This puts an immense power in the beholder of the most data as not only would you know how people are using the internet, but you would also have so many small bits of data around each visitor that you are now able to start to profile your visitor or future customer.

So how on earth does this relate to Piwik?

Well…..

Let’s suppose you are using another of the many large third party tracking programs, who do you think owns all of that valuable data that is being tracked from your site?

It’s certainly not you.  Sure you may be able to analyze that data and see some nice pretty reports, but at the end of the day that data does not belong to you and you are confined within the restraints that the third-party solution would put on you.

Now just imagine for a moment that you had access to a completely free piece of software that let you take control over your sites and the data collected from them….

Welcome To Piwik

Piwik is the ONLY software of its kind around that gives you total control over everything and the ability to expand and use your own data beyond your wildest dreams…

So on with the seven reasons.

#1 You Own The Data

This is one of the biggest reasons you would want to move to Piwik (this combined with #2 & 3) below and it becomes a knockout.

#2 Fully Expandable Feature Set Just Like WordPress

We all know that wordpress out of the box is nothing special.  It’s just another blog solution, until you start adding in plugins to extend functionality.  You could almost say the same about the Iphone, it’s the applications that make it.

Well this is one feature that makes Piwik REALLY shine out, the developers when setting out to build the ultimate free open source tracking system developed it with expansion in mind and built into it a plugin system that lets you just tap on entire new modules of function with simply a few mouse clicks.

This is HUGE when combined with #1 & #3

#3 API Developer Access

Complete API access is something that has been built into Piwik right from the very core.  This means that you can access any data from within the current functions of Piwik and easily import it into your own application to even further enhance your experience.

If you need to access more data or access it differently then you can easily create a new plugin from #2 to handle and work with that data, this will then make it available as an API.

In case the alarm bells have not gone off in your head just yet let me put this in another way.

Let’s say that you wanted to be able to see the top 3 pages viewed by visitors by country.  No worries all you need to do is create a new plugin that adds that functionality in and hey presto you can now do this.  But taking this even further.

With the combination of points #1, #2 & #3 you can now create your own custom applications (adobe air, .NET, php or any other language you like) that can import any of this data and then use it however it likes.
Imagine if you could have a desktop application that when you opened it could show you the following at a glance.

1.    How much you had made today in sales & from what products.
2.    A comparison of today vs the same day last month.
3.    The weekly average of your sales also in comparison to last week.
4.    Your top performing traffic sources by ROI (because you custom created a plugin that let you set a specific traffic referrer by advertising budget with them)
5.    The top pages that converted into a sale for you

This is just a simple example of how you now have an ROI & Commerce relevant application based entirely off your Piwik data.

Amazing right?

(Feel free to get in touch with me about developing custom Adobe Air applications I am looking for someone to give us a hand).

#4 Real Time Tracking

As its on your own server you can refresh stats as much as you want and get up to the second traffic stats.

#5 Total Privacy

Do you really know what all of these third party services are doing with your data?  What if it were in any way able to help your competitor gain an advantage over you?

Not anymore you have Piwik

#6 It’s Free

Hey free is always good, just this good has sugar coated on top.  Of course it is important to note that there are server costs or hosting costs though if you are running a few websites this is already covered.

#7 It Will Make You More Money

If data is valuable, and you now own all of your data, by default you have increased your wealth digitally.  You can now very easily run custom search queries or strings across all of your sites, across all converted goals or products and find trends that you may never have even noticed.

Piwik being only 1 year old and already covering as much ground as it does really sets the future for what you can expect from Piwik.

As you can see Piwik really kicks butt and is a serious contender to the analytics game, one that in my opinion gives you more power than any of them.

I know it is giving us an edge over our competition by letting us develop custom applications and plugins that provide data in way that has never before been achieved.

Over the coming weeks I will be posting more Piwik guides on how you can leverage Piwik even further in a real environment with real sales coming through the door, these will be live case studies on sites making well over 7 figures.

Please if you have any queries or questions about topics you would like covered in this  Using Piwik Effectively, Marketing Series feel free to post them as a comment and I will take note of them all.

Cheers

Marc Lindsay

Piwik 0.4

- ADDED new piwik.js Javascript Tracking API; More information in the blog post and the Javscript Tracking documentation #355 (by Anthon)
- ADDED search engine links now link to the search engine page for this keyword. Added definition for the main search engines; you can contribute more! if a definition, (eg. “search?q={k}” for Google) is not found, we simply link to the search engine homepage. More information how to add the link to the search result page on the Search engine FAQ #747 (by matt)
- UPDATED languages: Catalan, French, Danish, Hungarian, Ukrainian, Spanish, German, Taiwanese
- ADDED new languages: Czech, Thai, Finnish; Piwik is now translated in 27 languages
- ADDED 20 search engines (including the new Bing.com)
- FIXED #690 mysql password html encoded causing problems with special chars like & (by matt)
- FIXED #159 tooltip doesn’t show up on truncated labels with image (by black silence & Anthon)
- FIXED #691 Converted absolute URLs to relative URLs (by Anthon)
- FIXED #712 Updating to a new Piwik version should automatically force the browser to update the cache for JS/CSS files (by Anthon)
- FIXED #743 Optimized include path (by Anthon)
- FIXED #744 Killed Java detection – it wasn’t working properly in all browsers (by Anthon)
- FIXED #632 ZF classes declared twice when calling piwik api from a ZF project (by Anthon)
- FIXED #741 Updated to smarty 2.6.25 (by Anthon)
- FIXED #668 piwik_log_visit.location_ip is negative on 32 bit systems (by Anthon)
- FIXED #551 “Logout” / “Sign out” action linked to the active authentication plugin. (by Anthon)

Piwik 0.2.37

- FIXED #714 by deleting all dashboard entries that had the last visit graph widget. This should fix most issues and greatly minimize support (thanks vipsoft for the idea).

Piwik 0.2.36

- FIXED #706 – use single quotes for consistency; fix typo; upcase names
- FIXED #714 Now clearing Smarty templates on Update. Also fixing error with non-native json_encode() that is broken on some systems.
- FIXED #717 The dashboard layout string is too long for a GET request, now POSTing the data.
- FIXED #689 Added few search engines and fixed detection for google.ch
- FIXED #715 Added Arora browser detection

Piwik 0.2.35

Important note: After updating to 0.2.35, please delete your browser cache and refresh Piwik. There is a known issue where the new Javascript files are not correctly updated in the browser cache which causes the dashboard to fail.

- ADDED Refresh and back button now work! Great contribution by Khanh Pham #103
It is using the jquery history plugin. period, date, idsite, module and action are persisted across requests, making Piwik much nicer and faster to use.
- ADDED search field below data tables is now using the regular expression syntax. For example, a search for “google|yahoo” would match all rows containing “google” or “yahoo”. All search strings containing any of the special characters from this list: . \ + * ? [ ] $ ( ) { } = ! < > | must be escaped with a back slash, eg. if you want to search for keywords containing “piwik!” you would search for “piwik\!”.
- ADDED Introducing the new “ExampleUI” plugin which shows how to plot data easily in Piwik, reusing the existing framework (evolution graph, pie chart, bar graph, sparklines, tag clouds, tables, and more!)
- ADDED #126 all dates should be correctly displayed in all graphs. For example, evolution graph for days would show, on the X axis “Mon 29″, “Wed 31″. For months it would show “Aug 2009″, etc.
- ADDED Now displaying prettier date (better usability) as well as showing a calendar icon.
- API CHANGE method VisitFrequency.getSummary renamed in VisitFrequency.get for consistency with VisitsSummary.get
- API CHANGE: the API for the function Piwik_AddWidget has changed. The new API is Piwik_AddWidget( $widgetCategory, $widgetName, $controllerName, $controllerAction, $customParameters = array()). See examples of calls in all the core Piwik plugins. This change was necessary to make widgets more modular (they now accept custom parameters).
- API CHANGE: a small number of CSV outputs for some API calls would change following the simplification of DataTable_Simple implementation. Affected calls are VisitsSummary.get, Goals.get, VisitFrequency.get. This is due to a change in the implementation of DataTable_Simple (we simplified implementation).
- RENAMED time_before_archive_considered_outdated in time_before_today_archive_considered_outdated
- FIXED #84 Added proper translations for all columns, in tables, and graphs.
- FIXED #322 piwik is now using open flash chart 2
- ADDED: when hovering any of the sparklines, the UI makes it clear that clicking will refresh the evolution graph. This feature was in Piwik for months, and even Google Analytics implemented this UI feature after Piwik. However in Piwik it wasn’t clear to the user that the sparklines were clickable.
- ADDED: now widgets can be created with custom parameters. This makes it possible to create a widget that calls a controller->action with other custom parameters, this is used in Piwik to draw an evolution graph (module=VisitsSummary & action=getEvolutionGraph) for a given metric (&columns[]=nb_visits). These custom parameters are automatically forwarded to the sparkline url, the flash graph when clicked on sparkline, etc.
- ADDED The widget layout is now saved as a JSON string rather than a custom data structure. The dashboard code should be able to read & restore most of the layouts from the old format (except the evolution graphs widgets). Simplified the Dashboard.js, widgetMenu.js, cleaned up what was a messy code.
- ADDED sentence in Widgetize to let users know they can easily export the Piwik dashboard in an iframe.
- CHANGED the way translations used in Javascript are loaded: all translations strings finishing by _js will be loaded to be used in the templates when calling {loadJavascriptTranslations plugins=’YOUR_PLUGIN_NAME’}
- CHANGED Moved all templates in plugins under plugins/$PLUGIN/templates/
- CHANGED ‘Khtml (Konqueror, Safari)’ now displayed as ‘KHTML (Safari, Chrome)’
- FIXED #651 by setting session.save_handler explicitly (patch by vipsoft)
- FIXED #646, setting cache headers explicitly to nocache instead of assuming PHP default. (patch by vipsoft)
- FIXED #682 Notice: Undefined property: Piwik_DataTable_Simple::$rows
- ADDED new configuration option: default number of rows returned in API responses “API_datatable_default_limit = 50″
- REMOVED the automatic generic filters. The limit and sort and safe decode are applied by each module when necessary.
- REMOVED exact match filter. Now all searches are using regular expressions syntax. Exact match can be done using exact search ^like this$
- FIXED notice when natural sort on a non existing column
- FIXED CSV export for datatable_array
- CHANGED Clarified code for plotting multiple lines in an evolution chart
- FIXED #624 Added icon “save as image” below all graphs (next to the Export icon)
- CHANGED Moved all JS functions into the piwikHelper static class
- FIXED Added message when flash is disabled and graph not showing, linking to piwik faq.
- FIXED Added expressInstall.swf feature so that users using old flash version are pointed to the new one automatically
- FIXED #590 – use relative path
- REMOVED quicktime detection refs #683
- ADDED new option default_period to specify which period to use by default in piwik (default is day). Possible values: day, week, month, year

Piwik 0.2.34

- FIXED #648 Generating cache files fails resulting in no visits recorded for a few users, due to wrong order in includes in piwik.php

Piwik 0.2.33

In this release we focused on fixing all existing issues. At this point we believe Piwik should work very well and display accurate data. One step closer to Piwik stable!

- FIXED #374 Performance: Fix memory leak during Archiving process
- Improved overall speed of archiving algorithm
- ADDED Piwik now translated in Danish (Dansk), Greek (Ελληνικά), Estonian (Eesti keel), Galician (Galego) and Romanian (Română)!
- UPDATED translations for Catalan, German, French, Russian, Slovak, Swedish and Taiwanese.
- FIXED #310 When database collation is not the default, mysql error “Illegal mix of collations”; now all tables are created with utf8 charset. All existing tables and all fields are converted to utf8.
- FIXED #599 Make archive.sh not accessible via http; patch by pebosi and vipsoft!
- FIXED #589 Piwik fails to properly decode and store some chinese keywords (eg. from baidu.com)
- FIXED #435 Exotic encoded keywords should be stored as utf-8 in the DB
- FIXED #575 Duplicated keywords in search engine keywords report
- FIXED #577 Automatic purge of out of date records should be automatically called once a day.
- FIXED #640 Remove ‘nb_uniq_visitors’ from all API calls when period is not “day” as the data is not correct. It is now called ‘sum_daily_nb_uniq_visitors’ in all API calls. The column “Unique Visitors” won’t be displayed in week/month/year reports.
- FIXED #578 Actions API: clarify idsubdatatable VS databaseSubtableId in API response
- FIXED #582 API UsersManager.* accepting $userLogin should work well when super user login is passed
- FIXED #627 misreported outlinks (anthon)
- FIXED #408 recognizeTheVisitor doesn’t save new visitor if cookie is set and visitor not found in DB
- FIXED #633 when specifying logging messages/errors/exceptions to a file in piwik config, the archiving breaks as path is invalid
- FIXED #636 Anonymous access to javascript code
- FIXED #523 adding website with international characters doesn’t work; eg. täst.de
- FIXED #609 Misaligned data when exported in CSV
- FIXED #602 API: UsersManager.userExists returns empty value to mean no
- IMPROVED feedback to user during installation, in case user tries to install and conflicting tables are already found in the DB
- FIXED #612 Error when no sites configured
- FIXED #600 warning during archiving
- FIXED #597 Add new method to languagesManager to show all string of a language, to reuse piwik translations in other Piwik UIs (eg. http://www.desktop-web-analytics.com/)
- FIXED #581 add widget by default: RSS feed of changelog + refactored code it’s now very easy to create a widget showing a rss feed in any piwik plugin by reusing the Piwik_ExampleRssWidget_Rss class
- FIXED #606: custom vars broken (anthon)
- FIXED #586 claryfing messaging when one-click update fails
- FIXED #591 VisitsSummary index does not properly translate time period
- FIXED #592 not show example API link for SitesManager.updateSite() and addSite
- FIXED #593 Zend require_once path fix (anthon)
- FIXED #573 Changeset 949 may break configuration values
- FIXED #486 Sparklines aren’t distinguised by referer type
- FIXED #9 referer URLs with host registered as main_url or alias_url should NOT be counted as referer. Now if a visitor comes to piwik.org/ with a piwik.org/xx/y/z referer, it will be considered a direct entry rather than a referer visit. It makes use of the new caching mechanism introduced with the Goal Tracking plugin, to cache website-related data for super fast access at Tracker time.
- FIXED #14 Searching on countries within the DataTable is not working properly
- FIXED #614 MySQL unix socket support (anthon)
- FIXED the “Base table or view not found” or doesn’t exist (piwik_option) issue that was submitted by several users
- FIXED #605 error after installation: user translation tries to use ‘access’ before it’s loaded
- FIXED #603 API: when not specified, XML should be returned, rather than serialized PHP

- FIXED when profiling is enabled “Fatal error: Call to undefined function memory_get_usage()”
- API CHANGE we renamed two php constants. ENABLE_DISPATCH has been renamed PIWIK_ENABLE_DISPATCH and ENABLE_ERROR_HANDLER is renamed PIWIK_ENABLE_ERROR_HANDLER
- Renamed all constants used in Piwik and prefixed with PIWIK_ (no namespace collision when using piwik as php library)
- Added few search engines
- Other minor updates & improvements & added unit tests

Piwik 0.2.32

- FEATURE CHANGE previously, it was possible to login using the md5 hash of the password as the password, but for best practises and increased security we removed this feature.
- FEATURE CHANGE previously, it was possible to login using the “one click logme” as described in the FAQ; we added the rule that this method cannot be used to login as the Super User.
- API CHANGE slight change in the API of UsersManager.getTokenAuth($userLogin, $md5Password); previously, the second parameter could be either the password or the md5 hash of the password. For increased security and consistency, the second parameter is now required to be the md5 of the password. Please call md5() on the string before calling this API method.
- FIXED #571 (Filesystem access equals root access)
- FIXED clearspring “Export Anywhere” functionnality, click on Widgets on the top of the page, select a widget and click on “Export Anywhere”
- FIXED #561 (Piwik_DataTable_Renderer doesn’t use the Piwik_API_Request $request parms)
- FIXED #387 (admin user should not be prompted for super user pwd when clicking on admin link)
- FIXED #481 (username should be enclosed in double quotes in config files, in case the user name contains a =)
- FIXED #377 (Installation: check for php-xml: undefined function utf8_decode())
- FIXED #522 (unconsistency in login string length)
- FIXED #454 (Internal API incorrect results)
- UPDATED german and french translations
- ADDED Iphone, blackberry icons

Piwik 0.2.31

- ADDED One Click Update Mechanism. When a new Piwik version is out, the UI will link to a page that gives two options: automatic one click upgrade, or manual download. If automatic one click update is selected, Piwik will download piwik.orghref=”http://builds.piwik.org/latest.zip“, unzip, do a very basic check on the unzipped files, backup the configuration file and copy all files over existing files. For some configurations where the permissions (CHMOD) are not high enough, the one click ugrade may not work. This mechanism should help driving Piwik update rates from existing users, minimising impact of potential security issues, and maximising usage of latest piwik features!
- FIXED: Fatal error: Call to undefined function stream_context_create()
- FIXED #466 Support https reverse proxying. Patch by vipsoft
- ADDED few search engines
- FIXED #550 Widget + IE + ssl does not go together. Patch by vipsoft
- IMPROVED The language to country detection algorithm is much more robust, and the code very clean, added unit tests. Patch by Anthon! (see #504)
- FIXED #566 Broken Outlinks – Prototype JS breaks use of “for” in iterating over Array contents. Patch by Anthon
- FIXED #26 The user config file cannot only define a variable, it has to define the whole “category” of variables

Piwik 0.2.30

- ADDED detection of: windows 7, IE8, palmOS, iPod, blackberry, Android, iPhone
- ADDED automatic update check mechanism: once a day, piwik will check to see if a new piwik release is available. If a new release is available, the top right orange message will display a message and link to the new release and changelog.
- ADDED piwik users can now directly subscribe from the newsletter during the installation process; a quick and easy way to stay up to date with security updates, new plugins and new features (maximum 1 email per month)
- FIXED Javascript error when selected month is August (#374)
- FIXED following values were wrong in the database: idaction_ref & time_spent_ref_time
- FIXED piwik doesn’t require ctype extension anymore

Piwik 0.2.29

- FIXED “Exclude low population” in widget “Pages” does not do anything
- FIXED heuristics for detecting unique visitors was disabled in 0.2.28 (see enable_detect_unique_visitor_using_settings) ; unique visitors counts were slightly higher than normal.
- FIXED User Interface issues: when website selector is too long and was behind flash content (Patch by Anthon)
- FIXED bug when adding an already present widget in the dashboard (Patch by Anthon)
- FIXED path issues when using Piwik API from within a PHP script loading piwik files (Patch by Anthon)
- FIXED forcing a click as download does not work when the link has multiple CSS classes (Patch by Anthon)
- FIXED when using Piwik API internal, piwik forced the error and exception handler to be Piwik custom error/exception handler. Users can now define(‘ENABLE_ERROR_HANDLER’, false); to disable Piwik custom error/exception handler
- IMPROVED piwik.js features: Wildcard support in piwik_hosts_alias, option to set css classes to ignore automatic download/outlink tracking; see javascript tracking documentation
- ADDED hungarian language: thanks Viktor Horvath!
- ADDED option in configuration file to set cookie expire time for the tracking cookie and the login cookie. See login_cookie_expire and cookie_expire
- ADDED option in configuration file: show_website_selector_in_user_interface ; if set to false, the website selector will not be displayed in the Piwik UI. If your Piwik installation has thousands of websites, you may disable the website selector as it slows down the loading of Piwik reports
- ADDED option in configuration file to specify the email and name that appears in the email sent for password recovery; see login_password_recovery_email_address and login_password_recovery_email_name
- ADDED various search engines
- UPDATED most libraries used in Piwik: Smarty, jquery, swfobject, sparklines library, some Jquery plugins (Patch by Anthon, thanks pebosi for testing patch)
- UPDATED Dutch, Polish and Taiwanese language files

Piwik 0.2.28

FIXED For some PHP configurations, there was the warning “Warning: Call-time pass-by-reference has been deprecated;”. Fixed.

Piwik 0.2.27

- NEW Introducing beta release of Goal Tracking Plugin! Define your goals and see how your website is performing. A Goal can be a page view (or group of page), file download or click on an outlink. You can set an unlimited number of goals. Access Piwik reports to see who are your visitors that convert the most. Various segments are available: conversions by search engine, keyword, campaign, website, country, conversions by hour, by returning visitor or new visitor. Access your conversions report in one click on the icon below most tables in Piwik. Access all segments data by the new “Goal” tab in the Piwik UI. The Goal data is also available using the Piwik API (conversions, conversion rates, revenue, for each goal, for each segment).
This is beta release, there are known bugs and more features we will include, but we release it today to see what you think about it, and how we can make it better! We welcome your feedback!

- NEW Adding 4 new languages: Norwegian (by QTSystems), Slovak (by Zdenko Podobný), Serbian (by Petar Benke), Swedish (by Fredrik Astrom)
- IMPROVED Updating language Catalan, French, Taiwanese languages
- IMPROVED You can now select RSS export of all tables in Piwik, by clicking on the export link below the tables. Available formats are: rss, json, php, xml, rss.
- IMPROVED various improvements, simplification, and refactoring of the Piwik Core code, moving to a simpler architecture to help developers build plugins easily! Also introducing web tests using canoo (thank you Krystian!!)
- FIXED Password reminder now works when trying to reset the Super User password.
- FIXED Typo in the select list of months (december was “undefined”)
- FIXED Google Chrome now appears in KHTML family instead of Mozilla
- Piwik tracking cookie is now set to expire after 2 years.
- NEW Added new option in config file: default_module_login ; When loading piwik interface, Piwik will load by default the CoreHome module. You can override the setting to force the user to login.  This is useful when you have some websites view “anonymous” access but you want to force users to login instead of viewing the first anonymous website available.
- NEW Added new option in config file: enable_detect_unique_visitor_using_settings; if set to true, Piwik will try to match visitors without cookie to a previous visitor that has the same configuration: OS, browser, resolution, IP, etc. This heuristic adds an extra SQL query for each page view without cookie. It is advised to set it to true for more accurate detection of unique visitors. However when most users have the same IP, and the same configuration, it is advised to set it to false.

Piwik 0.2.26

- FIXED bug recently introduced where the Actions report would use too much memory

Piwik 0.2.25

- IMPROVED archiving is now more memory efficient; datatable are freed more often from memory. Users that experienced the “Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 32 bytes) in core/DataTable.php on line 822″ will be delighted
- IMPROVED archiving should be slightly faster. the job that deletes all out of date archives is now executed only 1 time out of 15 executions, but will eventually be done asynchronously.
- FIXED an important bug in the Website referrer detection: before not all unique URLs were properly archived. Now your Referrers > Websites > Click on any website domain > should show all data, as expected
- FIXED bug when installing Piwik without the table_prefix ; “option” is a reserver mysql word and there was issue with the Piwik table “option”. now fixed.