Roadmap 2.0

To achieve the goals set out in our mission statement, the Piwik team along with the Piwik community have defined a future vision for Piwik 2.0, as well as an extensive list of features we want to make happen.

This page details what our future goals are…

Announcement: Help us reach Piwik 2.0! We vitally need your financial sponsorship in order to make Piwik 2.0 a reality. Your funding will enable us to implement these new features much sooner. See below for more information on how you can sponsor a feature, or even crowd fund our best new ideas!

Piwik's mission statement is:

To create, as a community, the leading international open source web analytics platform, that gives every user full control of their data.

Data Tracking Improvements

Web Form Analytics – Form usage, abandoned, submitted
#562
  • Opened: December 31, 2012

    Modified: December 31, 2012

  • Priority: Normal

  • Owned by: matt

This feature is about adding in Piwik Form Analytics Reports, to let us know more information about how Forms are filled in, abandoned, and how can Piwik users improve their web form conversion rates! It would automatically track HTML forms in Piwik and produce reports in Actions> Forms.

We need your help, contact us to sponsor this feature!

New Report ‘Days to conversion/purchase’ (now available in Piwik 1.7!)
#2031
  • Opened: November 1, 2012

    Modified: December 14, 2012

  • Priority: Normal

    Component: Core

  • Milestone: 1.10 Piwik 1.10

    Owned by: matt

It would be interesting to know, when a conversion occurs, how many days ago the visitor landed on the website for the first time.

For example:

  • 1000 visitors converted to ‘Checkout’
  • 500 visitors, 0 days to purchase (bought on the first day!)
  • 100 visitors, 1 day to purchase (10% took one day to decide)
  • 400 visitors, 7 days to purchase (all others took their time, maybe this is something worth money, requires research, etc.)

See comment

New report: ‘Visits until conversion’ (now available in Piwik 1.7!)
#584
  • Opened: November 1, 2012

    Modified: December 14, 2012

  • Priority: Major

    Component: Core

  • Milestone: 1.10 Piwik 1.10

    Owned by: matt

We could provide a new report: visit count until conversion

Example
You have 1000 conversions

  • 500 conversions happened on the first visit
  • 200 conversions happened on the second visit
  • 60 conversions happened on the third to fifth visit
  • etc.
Support Google Analytics campaign parameters by default (implemented in Piwik 1.3!)
#855
  • Opened: November 1, 2012

    Modified: December 14, 2012

  • Priority: Major

    Component: Core

  • Milestone: Piwik 1.3

    Owned by: matt

Currently you can customize campaign parameters by editing config/config.ini.php and adding the following:

; variable name to track any campaign, for example CPC campaign
; Example: If a visitor first visits 'index.php?piwik_campaign=Adwords-CPC' then it will be counted as a campaign referer named 'Adwords-CPC'
campaign_var_name			= piwik_campaign

; variable name to track any campaign keyword
; Example: If a visitor first visits 'index.php?piwik_campaign=Adwords-CPC&piwik_kwd=My killer keyword' then it will be counted as a campaign referer named 'Adwords-CPC' with the keyword 'My killer keyword'
campaign_keyword_var_name	= piwik_kwd

New proposal:

  • each variable can be an array, so we can define multiple variable names
  • because GA defines 4 variable for campaign tracking (see URL builder), we would have to define Piwik campaign name, and keyword, by a concatenation of these. For example:
    # Campaign Name
    campaign_var_name[] = piwik_campaign
    campaign_var_name[] = "{$utm_campaign} {$utm_source} {$utm_medium}"
    
    # Keyword
    campaign_keyword_var_name[] = piwik_kwd
    campaign_keyword_var_name[] = utm_term

Note: as a more complex alternative, we can also contemplate adding a level in the campaign table, to handle 3 or 4 levels (for example, reusing the existing data table used for page views, that has infinite number of levels.

Asynchronous Javascript Tracking Code for faster website load (now available in Piwik 1.2!)
  • Opened: November 1, 2012

    Modified: November 1, 2012

  • Priority: Normal

  • Milestone: Milestone 2

    Owned by: matt

Include country & city detection via GeoIP database in core (Geo Locate your visitors is now available since Piwik 1.9!)
#1823
  • Opened: October 31, 2012

    Modified: December 15, 2012

  • Priority: Major

    Component: Core

  • Milestone: Milestone 2

    Owned by: User one

See doc: Geo Locate visitors countries cities and regions.

GeoIP plugin #45 is one of the most popular plugins. For a web analytics tool, getting user countries as accurately as possible is critical, and Piwik should help users in this direction.

When the plugin is released in trunk, we should update the FAQ, website pages and wiki pages mentionning GeoIP, and mark as closed the GeoIP ticket #45. For Goals compatibility of GeoIP plugin, see #1434.

Please let us know in the comments your feedback. If you would like to participate… well you know what to do!

New Report: Visitors who visited the website 1 time, 2 times, etc. (loyalty) (now available in Piwik 1.7!)
#536
  • Opened: November 1, 2012

    Modified: December 14, 2012

  • Priority: Major

    Component: Core

  • Milestone: 1.10 Piwik 1.10

    Owned by: matt

  • Loyalty: M people visited N times

See

We probably need to keep track of the count of visits for each visitor in a new table, to be able to tell this stat on more than 1 day range.

Create a Goal as ‘Spent at least N minutes’ or ‘Viewed at least N pages’
#2029
  • Opened: November 1, 2012

    Modified: December 14, 2012

  • Priority: Normal

    Component: Core

  • Milestone: Feature requests

    Owned by: matt

#reate Goals based on Time on site for visitors (eg. spent more than 5 minutes), as well as ‘Pages per visit’ (eg. Visited at least 5 pages)

Provide User documentation to use campaigns to track newsletters, marketing campaigns, adwords, etc. (Analytics campaigns & URL Builder now available in the docs!)
#604
  • Opened: November 1, 2012

    Modified: December 14, 2012

  • Priority: Major

    Component: Documentation

  • Milestone: 1.x - Piwik 1.x

    Owned by: matt

Support visitors with IPv6 (now available in Piwik 1.4!)
#1111
  • Opened: November 1, 2012

    Modified: November 1, 2012

  • Priority: Critical

    Component: Core

  • Milestone: 1.4 - Piwik 1.4

    Owned by: vipsoft

This is a follow-up ticket to #1110.

To support IPv6 addresses will require data conversion in the database:

  • IPv6 address are 128-bit. IP addresses are stored in the MySQL tables as 8-byte (64-bit) BIGINT UNSIGNED, or 4-byte (32-bit) INT UNSIGNED (as of r2237).

In php.net/ip2long, there’s an implementation of ip2long6 and long2ip6 that relies on the GNU MP (Multi-Precision) extension (‘gmp’). But for greater cross-platform support, we might also consider comparable implementations using the BC Math (‘bcmath’) extension.

Above uses inet_ntop and inet_pton to aid in conversion to/from user readable and internal representation (in_addr). (Note: Windows support added in php 5.3.)

Issues/Considerations:

  • Exclude visits from IP (added in Piwik 0.6)
  • on Windows XP + IIS7, $_SERVER['REMOTE_ADDR'] for “localhost” uses the deprecated form, “::1″
  • IP-to-country database/API support for ipv6
Implement First party cookies (now available in Piwik 1.1!)
#409
  • Opened: November 1, 2012

    Modified: November 1, 2012

  • Priority: Critical

    Component: Core

  • Milestone: Milestone 2

    Owned by: User one

Currently Piwik is using several third party cookies. we want Piwik to create, by default, 1st party cookies only. This is mainly for privacy reasons, but also for better accuracy in counting unique visitors (1st party cookies are more often accepted and less often deleted by users)

This ticket is a requirement for #134 and #1984

Tracking & Reporting Web Pages Speed and load time by URL
#1700
  • Opened: October 31, 2012

    Modified: December 15, 2012

  • Priority: Major

    Component: New Plugin

  • Milestone: Milestone 2

    Owned by: User one

The spec (formerly the Web Timing spec) hasn’t been finalized at this point.

http://dvcs.w3.org/hg/webperf/raw-file/tip/specs/NavigationTiming/Overview.html

See windows.msPerformance in IE9 preview, windows.webkitPerformance in Chrome 6. (windows.mozPerformance appears to have missed the feature freeze
deadline for Firefox 4.)

See: IE9 test drive demo visualization
http://ie.microsoft.com/testdrive/Performance/msPerformance/Default.html

High Traffic Piwik - Optimizations

Faster & Reliable Tracking: implement piwik.php asynchronous tracking import from logs every N minutes
#3632
  • Opened: December 21, 2012

    Modified: March 3, 2013

  • Priority: Major

    Component: Performance

  • Owned by: matt

Goal: by replaying piwik.php access logs every N minutes, implement piwik.php asynchronous tracking import.

This is a performance improvement ticket.

  • When this is enabled, the webserver will return very quickly the transparent GIF. It will also log the piwik.php request in the apache server logs. It turns out this request contains (by design) the full set of information required by Tracking.
  • We should provide a new CRON script that will, replay the piwik.php server logs every 1 minute for example. We could make use of Piwik Bulk Tracking features to import ie. 500 lines at once from the log.

Summary:

Power users will be able to setup Piwik where the Mysql is not required for tracking.
A script will run that will import, every N minutes (for example N=1 or N=60) the Webserver access logs into Piwik.

This script will use our log analytics tool to import the piwik.php requests.

It won’t be as “real time” as before since loading logs is asynchronous, but could be set every 1 minute for near real time.

This will make Piwik tracking decoupled from Mysql, more resilient, faster, easier to scale.

Fix “All Websites” dashboard loads slowly when Piwik tracks more than 1,000 websites (now available in Piwik 1.7!)
#1077
  • Opened: November 1, 2012

    Modified: November 1, 2012

  • Priority: Critical

    Component: Core

  • Milestone: 1.7 Piwik 1.7

    Owned by: matt

Currently, the multi sites dashboard page render takes 6 API requests (ie. 6 mysql SELECTs) for each website displayed. If a Piwik install has 1000 registered websites, the page load will simply fail, because the php page would try and execute 6,000 queries which is not a good idea.

Instead, only one request should be made to select the 6 values necessary for each website. This query would therefore select 6,000 values at once which would be as fast as we can get with that many values.

This query would join the site, access and archive_* tables. See Piwik_Access::reloadAccess() and core/Archive/Array/IndexedBySite.php for the current behavior. I think the solution might be to have a new class in core/Archive/Array indexed by user?

“Ranking Query” to decrease archiving memory consumption and transfered data between MySQL and PHP
#3330
  • Opened: November 1, 2012

    Modified: November 1, 2012

  • Priority: Major

    Component: Performance

  • Milestone: 1.x - Piwik 1.x

    Owned by: capedfuzz

Based on the discussions in #766, this ticket is the place for code that limits the number of results returned from MySQL.

The idea is to create a class that wraps an arbitrary SQL query with more SQL that limits the number of results while grouping the rest to “Others” and allows for some more fancy things that can be configured via an instance of the ranking query class.

Provide a setting to automatically purge logs that are older than a threshold (eg. 60 days) (implemented in Piwik 1.5!)
#5
  • Opened: November 1, 2012

    Modified: November 1, 2012

  • Priority: Critical

    Component: Core

  • Milestone: 1.x - Piwik 1.x

    Owned by: matt

The information contained in the tables _log_* should be purged automatically and regularly. Keeping all time logs in a single table significantly slows down the stats logging process (mysql having to rebuild indices, selecting from this million rows table is time consuming, maintenance is hard, etc.).

The goal of this task is to provide automatic purge of the tracking logs, every day or every month and with optional backup in a yearly table (customized with UI settings).

Processing uniques over weeks / months without using logs
In #409, we will implement a cookie store mechanism based on a mysql lookup table. This table will contain enough data (idvisitor, ip, idsite, date first visit, date last visit) to process unique visitors over a week or a month

Log purge execution
The purge task would be triggered during the ‘Maintenance process’ (see #1184), and once a day maximum it would execute and try to purge logs for the day (or month) before.

  • read settings (purge every day? every month?)
  • ensure all archives have been processed for the day archives contained in the log tables. Weeks/months are not necessary to be processed as they are the sum of daily archives (and uniques can be processed using the cookie store rather than keeping the full visitor logs for the period)
  • purge tracker for all websites at once
  • backs up the logs into a yearly archive table
    • INSERT INTO log_X_backups_2010 SELECT * FROM log_X
    • one backup per purged table (log_visit, log_conversion, etc.)
  • record the auto increment values (result of SELECT MAX() from log_X)
  • purge logs (TRUNCATE TABLE)
  • inserts last auto increment in each table

New Super User admin settings

  • Purge traffic logs and process reports at least every day/month
  • Backup traffic logs into a backup table containing all logs for the year (table is piwik_log_visit_backup2010)?
  • Could also add admin setting for ‘Keep visitor cookies for 1 year’ (from #409)

A few interesting resources

Lightweight Piwik mode: Feature to delete all reports older than N days from the DB (ideal for shared hosts with limited DB space, available in Piwik 1.8!
#53
  • Opened: November 1, 2012

    Modified: November 1, 2012

  • Priority: Critical

    Component: Core

  • Milestone: 1.9.x - Piwik 1.9.x

    Owned by: matt

The main reason that people stop using Piwik on small websites is because the DB grows out of control. Often users have 25Mb or 50m for the whole Mysql database. After installing the CMS, it is common for Piwik to have only 20M of space. We should try to make it easy to run Piwik with basic history and keep DB space below 20M.

Proposal “Lightweight Piwik” feature

  • Display the current DB size. Reuse API code from: #3004
  • Have a new setting “General settings” to select number of months to keep data for
    • By default, keep all historical data
    • In the dropdown selecting the number of months, put estimate of DB size that will be freed
  • When setting enabled,
    • Delete old archive_blob tables that are older than N months
    • delete from all archive_numeric all rows except a critical metrics (visits, pageviews, ecommerce/goal metrics).
    • also delete all segment data, numeric and blob
    • Propose a dropdown to select number of months
      • For each month choice “delete older than [3,6,9,12,18,2 years,4 years,custom]“. Once entry is selected, refresh the space free “delete older than [18 months] (N Gb will be deleted, Total Piwik DB size will become MGb)”
    • When the feature is enabled, execute it as a scheduled task (safer for huge DBs). But on page reload, we could also display a link to let the user execute the purge code NOW (when in a hurry to get the DB size under control).

Note: it would be very nice to also get rid of #2805 – just one SQL query away after all…
PS: and this one too #3003

V2 of this feature (to be put as “feature request” ticket)

  • User could choose exactly which reports to purge.
  • Have a “less agressive” purge mode that would keep more data, but still delete a lot of useless old reports nobody would care about
    • The list would be pre-selected with our estimate of the most useful reports (eg. Keywords, Page URLs/Page Titles, Websites, Campaigns, All goal reports…) and delete all others. Call our selection “Recommended historical reports to keep”
    • The list would handle numeric and blob.
    • “Delete data for segmented reports” or “Keep reports for segmented data”
Make “All Websites Dashboard” usable with 20,000+ websites
#3124
  • Opened: November 1, 2012

    Modified: November 1, 2012

  • Priority: Critical

    Component: Performance

  • Milestone: 1.9.x - Piwik 1.9.x

    Owned by: capedfuzz

Currently the All Websites dashboard will work well when there are hundreds or even thousands of websites, but after a point it becomes unusable. It should, however, be able to scale to any number of websites.

There are currently many barriers to scaling this area, including:

  • Lack of server-side pagination means all site data must be queried at once.
  • The design of the Array Archive types will result in unnecessary object creation (in this case several objects per website).
  • There’s no way to archive metrics for all websites, so summary & evolution data has to be calculated when queried.

These issues will be addressed in this ticket.

Powerful Admin Settings

Lock down accounts by IP after N failed attemps at logging
#2888
  • Opened: December 14, 2012

    Modified: December 15, 2012

  • Priority: Normal

    Component: Security

  • Owned by: matt

Our security policy aims to make security a principal design factor in all that we do at Piwik. One aspect that is currently possibly problematic is that good old brute force attacks could be vector of penetration in Piwik (if eg. attacker knows the login and quickly tries all passwords).

We should provide a core mechanism that would lock out, for 30min for example, a user after N failed attemps. Settings could be changed by the Super User and feature would be enabled by default, lock 30 min out after 5 failed attempts.

Implementation proposal:

  • Record, using Piwik_SetOption, count of lockdown for each IP that fails to enter valid login / pwd combination
  • After N failures, lock IP down and refuse authentication (even if the combination is actually valid!).
  • Document as FAQ, linked from UI, the sql to delete all locked out IPs in case the SU was actually locked out and can’t wait.

See ticket #2888 for more details.

Improve Admin menus (design, UX, submenus)
#1552
  • Opened: November 1, 2012

    Modified: March 3, 2013

  • Priority: Major

    Component: UI - UX (tpl, js, css)

  • Milestone: 1.9.2 - Piwik 1.9.2

    Owned by: halfdan

I’ll be resuming work on the plugin repository after 1.0.

Currently, the admin menus are organized as tabs. For “Plugins”, I propose to add submenus:

  • Installed (the current screen to activate/deactivate plugins)
  • Available (plugins available from the repository; not visible if no Piwik_Http::getTransportMethod())
  • Updates (installed plugins/core with updates available for download; not visible if no new updates)
Improve Javascript Tracking Code admin screen: asynchronous code, other options
#1845
  • Opened: October 31, 2012

    Modified: March 3, 2013

  • Priority: Major

    Component: UI - UX (tpl, js, css)

  • Milestone: Milestone 2

    Owned by: matt

We would like to propose an advanced page to allow users to generate a tailor made javascript code for their pages.

Some options that this Javascript Tracking tag generator would include are:

  • display the asynchronous code only (currently we only display the non async code)
  • A radio button to “Group Page Titles report by subdomain”. Useful when tracking multiple subdomains in the same Piwik website. See #1657
  • A radio button to “Track unique visitors across subdomains of $WEBSITE_HOST”. Would call .setDomains(‘.example.org’);
    • This would also ensure clicks to sub domains are not tracked as outlinks.
  • Should we default to load the JS from /js/ instead of piwik.js? This has the advantage of using compression, expire headers, etc. will result in a faster experience for webservers that do not set expire headers on static JS files.
  • allow to customize the campaign and name and keyword parameters (similar to GA “My online campaigns from other providers use different tags than those used by Google Analytics”). Calls setCampaignNameKey and/or setCampaignKeywordKey
  • Have a separate tool / tab for “Image tracker” that would generate the image given a few possible parameters (page name, idgoal / revenue)
    • This would greatly increase visibility of image tracker feature and would allow marketer to easily tag their newsletters, press releases, ebay pages, with the image tracker.

Others:

  • Explain to users that they might need to escape the code when pasting in a template system such as Smarty
  • Explain that on dynamic pages, code can be added to a footer file or similar
  • Update the documentation at: http://piwik.org/docs/javascript-tracking/and other places, to use the asynchronous version
    • Could we use a JS plugin that allows to show both versions (async by default, tab above the code snippet allowing to switch to non async?)
  • Allow to enable “Ping on page” that will improve bounce rate metric by pinging Piwik regularly when the tab is on focus. see #2041

http://www.tracewatch.com/doc/code

  • Finally, it would be nice to review the JS Code section of the Installer to: simplify the text, simplify the UI, give async by default. Also, it would be nice to finally i18n this text so that all users can see it in their language as they all go through it during install!! :)
Add ‘System Check’ page in Admin section
#2122
  • Opened: December 14, 2012

    Modified: March 3, 2013

  • Priority: Normal

  • Owned by: capedfuzz

The Piwik System Check tool is currently only displayed during the installation of Piwik. Once Piwik is installed it should still be possible to access this interesting report. For example, server configuration often change and it would  useful to see if a server is still compatible with Piwik at any time.

  • Add a menu entry in Piwik admin UI only for Super User (page loads only for SU)
  • The system check should display detected PHP and GD versions
  • When file integrity fails, clarify error message and make link “details” more obvious
  • The system check is already implemented and looks for all required php extensions and various other settings.

if any other quick fixes that we could tackle at the same time, please comment.

Scheduled report list of improvements
#3088
  • Opened: November 1, 2012

    Modified: December 15, 2012

  • Priority: Normal

    Component: Core

  • Milestone: 1.x - Piwik 1.x

    Owned by: JulienM

This ticket is a placeholder to improve the current feature of Scheduled email reports (PDF and HTML).

See the following tickets for new feature requests:

High impact features:

  • #2012: support ‘Yearly’ report
  • #2197: Add possibility to unsubscribe from the “mailing list”
  • #3014: “Referrer Type” could have a graph plotting each type over last periods (would need support for graph with multiple lines which would be great to have for Piwik Mobile of course!)
  • #2762: set day of week to send and set expiration date
  • #2137: new Option to expand and show the hierarchy for Pages/downloads /Custom variables/etc. reports
  • #2817: New option to include all rows (no limit)
  • #3336: Scheduled reports “View All” feature

Please comment if you have other ideas to improve Scheduled reports. Contact us for sponsoring and donations.

Ultimate Analytics Platform

Organize and Display Websites by Group in ‘All websites’ dashboard
#2174
  • Opened: December 14, 2012

    Modified: December 15, 2012

  • Priority: Normal

  • Owned by: matt

For users tracking 10, 20 or more websites, often websites belong to groups (for example “Intranet”, “Blogs” or “Ecommerce sites”), groups can be the type of the website, a given project which contains several websites, or an agency would want to organize websites by customer.

In the API, it is already possible to assign a ‘group’ to a website. However this functionnality is not available in the User Interface yet. The Piwik UI should allow to create groups, assign sites to groups, and show aggregate stats for each group.

Specification:

  • In the ‘All Websites’ dashboard, the Super User could see an aggregate number of visits/pages for the group. Websites would appear below their group row.
  • the Websites edit table will have a new column when Super User only, displaying the group the website belongs to.
  • the website selector would also show the website groups, if possible somehow?
    • search would also search for groups
    • When group is matching for a website, display it in the search result along with the group name to show the group matched the search.
    • Sites belonging to a matched group should show as a nested list under the group name. If a site & its group both match the search, there should NOT be a duplicate result.
  • would be nice to only have this feature for “advanced” users (especially users with 5+ websites or so?), therefore maybe a new setting would be required to enable this, for example in “General Settings > Enable Websites Groups
  • when there are 5+ websites in Piwik, in “All websites” write in grey below the table “You can assign websites to Groups. Enable the feature here”. or better msg?
New feature: “Display Trends and Analytics summary”
#3032
  • Opened: December 14, 2012

    Modified: December 14, 2012

  • Priority: Major

    Component: User Interface - Analytics reports

  • Owned by: matt

The goal is to propose a new view into for existing reports in Piwik, mining the underlying data in relevant ways to give the user a summary of what has changed in the data, what is working out well, and what is getting better.

Content of the new report

The new UI would display a few new reports to give insights about what is worth keeping on an eye on:

  • Best new [Websites] in the selected period
  • Best new [Websites] in last 3 periods
    • NB: new website are websites that have traffic in the period and was not used in the previous 3 periods before this one
  • Fastest growing [websites] contributing to overall traffic
    • NB: these are the websites that are not new, but are now contributing more than before, and have significant % visits
    • NB: The % treshold could maybe be modified with a simple controller — and recorded/restored

Notes

  • Table display: Each table for these reports would display between 0 and 6 rows by default,
    • the view would be filtered to only show the statistically significant changes that are worth looking into
    • Later in V2, allow user to modify the % threshold used to define “significant” changes. Record as setting and restore.
  • Metric used: The algorithm would be based on Visits metric by default for most reports. For the tables in the “Goals” report, the algorithm would use the number of conversions as metric.
  • when there is no visit to compare against (new website), be clever and select the most oldest data available and show trends on what’s there only (or show a message “No trends yet” or similar when not enough data).
  • the new Analytics view would be available on most reports in Piwik: Keywords, Websites, Page URLs, Page Titles, Browsers, Countries, etc. displays when a new icon is clicked. Icon would be a light bulb for example “Trends” below all reports http://www.iconfinder.com/icondetails/25348/22/
    • or displayed by default in some reports, for example on the right space currently empty in most report, next to the: Keywords report, Websites, Campaigns, Page titles, etc.

There would be a metric selector at the bottom of these reports, to choose the metric (Visits VS Conversions VS Ecommerce Orders VS Revenue etc.)

  • The Trends reports could also be added to PDF/HTML reports
  • For tables with subtables, eg. Custom Variables, Campaigns/Keywords, we could look at all subtables as well, to hilight the significant increases or new values (based on number of visits)

Feedback
Please answer these questions:

  • When you use Piwik, what are the regular things you are looking for?
    • List the reports or mental checks you do
    • Can these be automated in Piwik somehow?
Show graph of visits/pages over the last 30 days, for a specific row (keyword, country, etc.) see the new Row Evolution screenshots
#534
  • Opened: November 1, 2012

    Modified: December 15, 2012

  • Priority: Critical

    Component: UI - UX (tpl, js, css)

  • Milestone: Piwik 1.8

    Owned by: matt

Show graph of visits/pages over the last 30 days, for a specific row (keyword, country, etc.)

The plugin would add, in all data tables, an icon next to the keyword/ country / page view (or on hover), that would ajax load an evolution graph of the number of visits for this keyword, country or page, over the last 30 days/weeks/month. The graph would have the ‘data export’ links.

UI

We should find out how to provide this UI feature without bloating the UI with icons and new functionnality. Note that some rows in Piwik are already clickable (eg. to get the list of keyword by search engine).

One way of doing it would be to show a semi transparent icon when hovering over a row. The icon would be shown at the end of the row.

The new UI would also be used to report the new reports from #756

Report
The report would contain, for a specific keyword or search engine or country or page, the following information:

  • visits in date range + sparkline
  • unique visitors in date range + sparkline
  • bounce rate + sparkline
  • (for pages) time per page, exit rate, etc. + sparkline

Also, we should report Goals conversions, maybe in a different report, available by a click on a tab, or some other mechanism.
This report would show, for this keyword/country/page, the following info:

  • overall conversions / conversion rate / revenue per visit
  • for each goal…
    • conversions / conversion rate/ revenue per visit.

Performance note

Piwik stores data on a “per day” basis. When plotting number of page views for the url ‘http://example.org/category/photo‘ for the last 30 days, it will require selecting data for the 30 days, and navigate through the tree to get the child node data (/photo) – this is all done automatically via filters. In this example, piwik would select 90 blobs of data, and unserialize, load in memory, grep, etc. This is a severe requirement and feasability should be studied before starting the implementation.

When plotting evolution of keyword “piwik” over the last 30 days, this would require 30 blobs of data to be selected/unserialize/loaded/etc.

Note

In the first version, only one data would be plottable. In future versions, we could think about plotting 3 or 4 keywords/countris on the same graph. How would this be available in the UI? Clicking on the icon could “enable” the line to be displayed on the graph, and a “reset” button, or a click on the icon, would disable this line. Requires #397.

Open source SVG world map to show regions and cities – be 100% Flash free!
#1652
  • Opened: November 1, 2012

    Modified: March 3, 2013

  • Priority: Normal

    Component: Core

  • Milestone: 1.x - Piwik 1.x

    Owned by: greg

Right now the world map only shows which country the visits are coming from. It would be really great / useful if we could narrow it down to what state the visitor came from or even which city. Like in the Google analytics.

It’ll be a great feature for marketing so we know where most of the visitors are!

We would like to also move the technology from Flash to full open source stack, and have the map displayed in SVG or Vector.

Once the Maps show cities and region, we definitely have to show the maps in the actual Country report: full awesomeness! see #1821

Ecommerce Analytics (Analytics Ecommerce now available in Piwik 1.5!)
#898
  • Opened: October 31, 2012

    Modified: October 31, 2012

  • Priority: Critical

    Component: New Plugin

  • Milestone: Milestone 2

    Owned by: User one

This ticket is about defining the scope and new features of a Ecommerce plugin in Piwik.

The Ecommerce statistics feature is now built in Piwik, check it out!

Support 5 Custom variables at the visit level, and provide new reports on these variables (now available in Piwik 1.1!)
#1984
  • Opened: October 31, 2012

    Modified: October 31, 2012

  • Priority: Critical

    Component: Core

  • Milestone: Milestone 2

    Owned by: User one

FEATURE RELEASED in Piwik 1.2

See the documentation in: Web analytics – Tracking Custom Variables

The goal of this feature is to put visits into buckets (segments) via the Piwik JS API. Then access reports about these custom name/values just like other reports in Piwik.

Custom data: Use cases

  • Count Logged in users VS logged out visitors (or Customers VS prospects)
  • Count custom things per visit that are not included in Piwik standard metrics, eg. ‘games played’, ‘comments left’, or anything etc.
  • Track position of keywords in google search page (using new parameter found in ref url recently after quick parsing of ref url in JS)

Custom data: New default reports

  • Custom Variable: list all distinct names, for each list visits, pages, time on site, bounce rate, etc.
  • On click on any variable name, list all distinct values for this custom variable with same metrics.

This new report could be displayed in “Visitors > Custom Variables”

Custom data: JS API piwik.js

  • new .setCustomData(index, name, value)
  • new .getCustomData(name)

We should allow for up to 3 custom variables (GA does 5 but 3 will be fine!).

This up to 3 (key,value) pairs will be stored in 1st party cookie.

For a given index, name should always be the same.

Scope of cookie is session scope (dies when browser dies).

The content of the cookie is passed to piwik.php (c1n=custom_name_1&c1v=custom_value_1 etc.?).

The name,value pairs should be limited in length to not excell the URL limit (they are send on each page view along with current URL, page name and possibly url ref and more).

Setting a custom var with the same slot in a subsequent page view will overwrite previous value set.

Also update PHP Tracker API with these new methods.

Custom data: Schema update

Add 6 fields to the log_visit table: custom_name_1, custom_value_1, custom_name_2, custom_value_2, etc.

name and value are the parameters to JS function piwik.setCustomVar(1, name, value)

Custom data: Tracker update

New visitor and existing visitor will set the custom variable if passed.

Custom data: Archiving

The new reports (see above) require couple new queries.

Custom data: API

New API function released to report the custom data names and values, similar to existing functions.

Performance impact

When no custom var is set, this should not affect the speed (ie. no performance regression in normal use case). Check this is the case.

For each custom variable, distinct values should be kept low whenever possible (eg. 10 or 20 disctinct values).

Nice to have

  • Update Live! widget and visitor log to show the custom variables set

Please submit any feedback/question.

Piwik as the ultimate Urchin alternative and AWstats alternative: modern web analytics from server logs! (now available since Piwik 1.8!)
#703
  • Opened: October 31, 2012

    Modified: October 31, 2012

  • Priority: Major

    Component: Core

  • Milestone: Milestone 2

    Owned by: User one

Urchin Alternative: Import your server logs in Piwik, the Free web analytics platform!

See blog post Piwik alternative to Urchin for more information.

Piwik is the Urchin alternative but also Webalyzer and AWStats alternative: with a Python script, you can now import webserver logs (apache, iis, and more) in Piwik, instead of using the javascript tracking.

Description
A Python script available in piwik/misc/log-analytics/ will parse server logs efficiently and automatically call the Piwik Tracking API to inject the visits/pageviews/downloads in Piwik.

How to install / how to use

  • Requires Piwik >= 1.7.2-rc2. Download the latest version from http://builds.piwik.org/?C=N;O=D
  • Requires at least Python 2.6
  • Requires one or many server log files, typically called access.log in Apache for example. These log files will be imported into Piwik.
  • You can also create a “test website” in Piwik to import all data into, rather than importing into your existing websites. Then, use the command –idsite=X to force all info from the log files to be imported into this idsite
  • You can use –dry-run command to have a test run and make sure you will not track data or create new websites

SEE FOLLOW UP TICKET #3163

How you can help?

  • please use the script and report your feedback and bugs here
  • if you are a hacker yourself, please review the code and consider submitting performance optimization, or improvements.
  • If you are a webhost or web agency and wish to offer Piwik to hundreds of your customers, please contact us
  • review the doc at Server log analytics

Tasks to do before final release

  • Test, test and test
  • Setup on demo.piwik.org in a new website
  • Check all code review feedback managed
  • Review Import Logs in Piwik doc page.
  • decomission apache2piwik (update blog post)

Feature requests for V2 or later

SEE FOLLOW UP TICKET #3163

SEO: Report average keyword position for google keywords
#2466
  • Opened: October 31, 2012

    Modified: December 15, 2012

  • Priority: Major

    Component: Core

  • Milestone: Milestone 2

    Owned by: User one

In a percentage of requests from google, google gives the ranking of the result on the search page.

We currently parse this ranking and display it in the Live! or Visitor Log when we have it. (see function getKeywordPosition() in plugins/Live/Visitor.php)

It could be interesting to, add a new field in log_visit to keep track of this ranking when it is passed, and add a new column in the keyword report “Average position” of keyword.

Social networks tab and widget
#2791
  • Opened: October 31, 2012

    Modified: March 3, 2013

  • Priority: Major

    Component: Core

  • Milestone: Milestone 2

    Owned by: matt

Hello there!

We have done some work with the ‘Referrers’ plugin and added a new tab “Referrers”->”Social Networks”, where you can see entry counts and referrer link from many social networks.

And it works this way:

It requests to the website list (like it does for the “Referrers”->”Websites” tab) and after that it uses the ‘pattern’ filter.

The list of social networks is here: /piwik/plugins/Referers/data/socials.csv (55 snw)

Each social network gets related icon (like it does for the “Referrers”->”Search Engines & Keywords” tab) from the piwik/plugins/Referers/images/socials folder.

Also we created a new widget: ‘Referrers/list of social networks‘ and we find it very useful. You can add this on you dashboard now.

So that’s it.

We would like to make it able to view this social networks info also in the “Referrers”->”Overview” tab. But unfortunately it can’t be done using a filtering. So we just put it aside for a while.

If you have some ideas about all this social networks features — that could help to make it more useful — we would love to hear from you and try to make it real.

See the attachments for the core patch.

Best regards.
Codax team

Page insight analytics: page overlay showing where users click when they visit a particular page
#2465
  • Opened: October 31, 2012

    Modified: March 3, 2013

  • Priority: Major

    Component: Core

  • Milestone: Milestone 2

    Owned by: matt

In order to show analytics data as an overlay, we have to set up a cross-domain communication protocol. The aim of this ticket is to complete a simple version of the feature that provides extensible mechanisms for the communication. Then, we will implement the Overlay report showing the number of clicks over the website. It will be beautiful & super useful for advanced analysis.

Create custom alerts on any Piwik reports and metrics, and automatically receive an email when they are triggered
#1486
  • Opened: October 31, 2012

    Modified: October 31, 2012

  • Priority: Major

    Component: New Plugin

  • Milestone: Milestone 2

    Owned by: User one

Create custom alerts:

  • define the alert name (“Big drop in purchases”)
  • select a website (or list of websites) on which the Alert is defined
  • receive alert by email (email will contain alert description + link to Piwik dashboard URL for the given website ID and period).
  • select the Alert period: should it be daily, weekly or monthly?
  • select the report (Websites, Keywords, Countries, general stats)
  • define Metrics (visits, page view, avg. visit duration, Goal 1 conversions, total goal conversions, etc.)
  • define the Alert: when Visits decrease 50%, when purchases are more than 50 per day, etc.

See attached screenshot for preview.

Note: Autocomplete the dimensions will probably not be in the V1 of the script. Autocomplete would be a nice feature as it helps users not making mistakes (eg. is it Germany or DE or germany?).

This plugin requires each Piwik plugin to define their API metadata (list of reports callable, and list of metrics + translations for each metric). Covered in #1485

The plugin is being designed and implemented by halfdan.

New “Visitor Profile” report showing all info about a specific Visitor
#3089
  • Opened: December 14, 2012

    Modified: December 14, 2012

  • Priority: Critical

  • Owned by: matt

Piwik tracks detailed visitor information, and after a few months of use Piwik will contain interesting information about new and returning visitors. Currently, we display the list of visits in the Visitor>Visitor Log report, but there is no way to show a report for a specific visitor.

This ticket is about adding a new report specifically tailored to a visitor. THe report will show:

  • All visitor info
  • List of past visits and pageviews, downloads, outlinks, site searches, goal conversions, ecommerce conversions and product bought.
  • List the map showing where the user connected from
  • List the first/last visit information
  • List a summary of all visitor activity over time
  • Show a graph of the last visits

Also:

Create annotations in the Graphs and list all Text notes for a given date range
#1253
  • Opened: December 14, 2012

    Modified: March 3, 2013

  • Priority: Major

  • Owned by: matt

An annotation is a Text note set a specific date. Add the ability in Piwik to annotate reports, e.g., a spike in traffic attributed to a slashdot story.

These annotation would appear with a marker on the Graph, and would be listed below the graph on click on “Annotations” button, and could also be created/queried via the API.

Segmentation: implementation UI for adding/editing and switching segments
#2135
  • Opened: November 1, 2012

    Modified: December 15, 2012

  • Priority: Critical

    Component: Core

  • Milestone: 1.10 Piwik 1.10

    Owned by: matt

Piwik now support full segmentation via the API only: http://piwik.org/docs/analytics-api/segmentation/

We want to be able to

  • Create segments
    • define the conditions AND / OR
      • pick metric
      • pick match (equals, contains, etc.)
      • pick value
    • give the segment a name
    • Segment is created inside the “report” UI like Goals
  • Test Segment in real time (fetch number of visits for it). Just calls the API with segment= and fetches number of visits.
  • Edit segment: UI page resets to current segment definition
  • Build the feature to autocomplete when filling a value for a given dimension/metric, for efficiency and ease of use. Reuse the code in getLabelsFromTable()
  • For metrics, when the value in the filter is not numeric, throw an error

Also, when creating a segment, you can add check 2 boxes

  • Add to my segment list (checked by default – available only to user who creates it)
  • Add to this website segment list (viewable by all & editable by all users with admin on this site)
  • For super user, “Add this to all websites segment list”.

Segment definitions are stored in a new ‘segment’ table, which contains the creator user login, idsites the segment appears in.
In the super user case, the segment will have a flag “idsites” = “all”.

In the Reporting UI, there would be a new box probaby similar to current Calendar / Info box that would list the current Segment applied to the data.
It would open up (similarly to the calendar) to show the list of pre-created segments, or the “Add a new segment” link.

User can then click on the Segment link to ajax reload the report with the segment applied to all visible reports.

The list of segment will lists all Websites segment first, then all User segments.

Note: some of the UI logic to define segment can be similar to the one used to define custom alerts (#1486), in particular the “value” ajax suggestion would reuse the same code.

GA supports visitsToTransaction and daysToTransaction – is it equivalent to “visitConverted==1;daysSinceFirstVisit==X” ? maybe not

Summary: the UI will allow to create/edit/delete segments, we will add a new box next to the calendar to select currently applied segments.
Later version

In the V1, we will not compare the Segment data with the “All visits” data or allow to compare segments together. This feature would be very nice and is important to have, see also ticket to compare several date ranges #291

Requirements

I will ask Marcin to provide some mockups for the screens of this feature. Once we have them we can implement this new UI element.

Please post here any idea/feedback!

Allow customizing the Piwik logo and text (now available in Piwik 1.5!)
#2008
  • Opened: October 31, 2012

    Modified: October 31, 2012

  • Priority: Critical

    Component: UI - UX (tpl, js, css)

  • Milestone: Milestone 2

    Owned by: User one

We are probably not going to implement theming in the near future (#177), but a cheap and useful alternative would be to just allow ‘logo’ theming of Piwik. We would still mention ‘Piwik’ all over the place (too much work to do otherwise), but customizing the logo would be good enough for a start.

See also #1148

Implementation proposal: currently the Piwik logo is a text, but it could be changed to an image. There are a few places where it is displayed:

  • Login screen
  • Product header (reporting UI and Admin UI)
  • Error message (ie. set date=test in the URL)

There could be a new section in the ‘General Settings’ admin tab, that would allow to upload a .PNG or .JPG logo picture.

The image would then be resized to fit the product header, resized for the login screen and error messages (slightly different sizes).

The admin setting could be:

(x) Use Piwik Logo
( ) Use Custom logo <img src=’show current logo’> (when this is clicked, also display “Upload logo” which would overwrite the current logo if any).

The setting could be recorded in the piwik_option table. The logo would be renamed to custom-logo.png so there is no need to record the ‘custom logo filename’.

Provide custom date range feature (now available in Piwik 1.3!)
#572
  • Opened: October 31, 2012

    Modified: October 31, 2012

  • Priority: Critical

    Component: Core

  • Milestone: Milestone 2

    Owned by: User one

Piwik provides reports for days, weeks, month and year, but not for custom date ranges.

The mechanism used to process reports for weeks, month and year use a generic code that could be used to generate reports on any custom date range.

List of changes to make this happen:

  • the date range should be set in the “date” parameter and not in a new parameter date_range, and “period” parameter would be “range”
  • Archiving: when period=range, the Archiving code will test if an archive exists already. If it doesn’t, it will look at the most efficient way to process the range: the minimum number of sub date ranges.
    • For example, requesting data from Jan 1st to March 15th 2009:
      • we can sum jan month + feb + sum of the first 15 days of july? which is 17 archives to select and sum
      • but if we minimize the nb of date ranges in the requested range, by doing: jan + feb + day 1st of march + week from 2nd to 8th + week from 9th to 15th, we only select & sum 5 archives which will be a lot more efficient and better user experience.

The archive sum of all sub archives will then be recorded and read as normal.

  • UI: period selector will change to use a calendar that allows for date range selector (example). We could add “Date range” after “day, month, year” buttons, when clicked would show 2 calendars.
  • UI: maybe it would be nice to show date ranges shortcut, “last 7 days”, “last 30 days” somehow ?
  • QA: add new integration tests that test with the date range “date=2011-01-01,2011-03-22&period=range”
  • Note: none of the plugins code for Archiving, API querying etc. will need to be modified, since all the logic is hidden

Any ideas? Any volunteer to work on this interesting challenge? Please comment :)

New report “Visitors Recency”, days since last visit (now available in Piwik 1.7!)
#583
  • Opened: October 31, 2012

    Modified: October 31, 2012

  • Priority: Major

    Component: Core

  • Milestone: Milestone 2

    Owned by: User one

We could provide a new report: days since last visit

Example

You have 1000 unique visitors;

  • 500 are new visitors
  • 300 visited your site 1 day ago
  • 100 visited your site between 2 and 5 days ago
  • 10 visited your site between 5 and 15 ago
  • etc.
Send scheduled reports via SMS / text message (available in Piwik 1.8!)
#2708
  • Opened: October 31, 2012

    Modified: December 15, 2012

  • Priority: Major

    Component: New Plugin

  • Milestone: Milestone 2

    Owned by: User one

The idea is to send reports via SMS using online SMS providers. Users need to provide one or several phone numbers and a SMS authorization token.

Reports

So far, we have identified three different kinds of reports. Here are the candidate templates :

  • Standard Report for 1 website:
from=Report Virtual-drums.com Daily
text=Yesterday, 150 visits (+15%), 555 pages (+22%), Goals: X conversions (-XX%), Revenue: xx€ (-xx%).
  • Report “All websites”
from=Report Virtual-drums.com Daily
text=Yesterday (Jan 21st): virtual-drums.com: 150 visits (+15%), 555 pages (+22%), Goals: X conversions (-XX%), Revenue: xx€ (-xx%). forum.piwik.org: 600 visits (-12%), 32 pages (+2%), mamamia.fr: 330 visits (+1%), 555 pages (+22%), Ecommerce Revenue: 34$ (-60%) from 45 orders (+111%)
  • Information displayed for all websites: Visits, Pages
  • Information displayed for all websites that have at least one conversion: Conversions, Revenue (if Revenue > 0)
  • Information displayed for all website with Ecommerce enabled: Orders, Revenue
  • API to use:
    • When “All Websites” report is selected: The API MultiSites.getAll can be used for these reports
      • It needs to be enhanced to also return the number of Conversions, and Ecommerce orders.
      • To not add overhead for the “All websites” dashboard, maybe we could add a new parameter to MultiSites.getAll called “enhanced”, 0 by default. When enhanced==1, we would enrich the returning data with Conversions/Ecommerce orders?
    • When only “Standard summary report” is selected: Could we also use the MultiSites API, but implement it so it works for one website only?
      • The advantage of using this API is that it already returns the % evolution so that the SMS reports would not have to process any data
      • the implementation code for a new MultiSites.get could be shared with the existing MultiSites.getAll

Desirable features

  • The periodicity of SMS reports is configurable the same way as for HTML and PDF reports.
  • SMS reports can be downloaded for preview the same way as for HTML and PDF reports.

SMS Authorization Token and Account Management

Online SMS providers with open APIs provide a SMS authorization token to registered users.

Within Piwik, this token will be manageable one of two ways :

  • One token by Piwik instance

In this case, it is the responsibility of the super user to provide the token in the SMS management user interface.

It is the only user who has the right to consult the number of remaining credits.

  • One token by user

In this case, it is the responsibility of each user to provide the token.

Each user has access to the number of remaining credits.

Deciding between the two modes of management is done by the super user with the following options in the SMS management UI :

- (x) Allow other users to create and manage their own SMS account.
- ( ) Allow users with ‘view’ or ‘admin’ access to send SMS reports using your SMS credit.

The second option is the default.

When a token is typed-in by the user, a call to the SMS API will be performed to validate the token and display the amount of remaining credit. The token will not be stored if it can’t be validated.

The user will have the ability to display the number of remaining credits in the SMS management pages. A message will be displayed in the report and SMS management pages when no credit are left in the account.

The token can not be modified. To change an already validated token the user will have to remove it and add a new one.

If the token is not provided

  • In the SMS management page

When the token is managed by the super user and a standard user accesses the SMS management page, display “SMS reports are not properly configured, please contact your administrator”.

Phone Number

Within the SMS management UI, each user can manage one or several phone numbers. A phone number will be selectable within the report creation and edition forms only if it has been validated. To validate a phone number, a token is sent via SMS after requesting it in the SMS management page. This token is then typed-in by the user.

Here is the candidate SMS validation template :

Code is 54321. To validate your phone number and receive Piwik SMS reports please copy this code in the form accessible via Piwik > Settings > SMS.

The validation token does not expire.

Phone numbers can not be modified. To change an already validated number the user will have to remove it and add a new one.

The UI will provide a list of countries to automatically assign the international extension number. The country corresponding to the current language will be selected as the default value.

The phone number input field accepts only numbers.

A report can be created and edited without assigning phone numbers. The report list will display a warning message and the user won’t be able to send the report.

In the report page, when the user has not validated at least one number :

SMS reports are still editable, the form can be saved without a selected phone number but the following message will be displayed in the edit form: “The list of phone numbers is empty because you did not activate a phone number”. A link will be provided for quick access.

SMS Management Page

The SMS management page will be added in “Settings”.

Technical aspects

  • This SMS report feature will be developed as a new plugin called MobileMessaging. It will be included in core but disabled by default.
  • The code managing the SMS authorization token and phone numbers should be reusable by other features such as SMS alerts. Therefore, the MobileMessaging plugin should work without the PDFReports plugin.
  • PDFReports plugin templates will have to implement a hook mechanism.
  • Phone numbers and auth tokens will be stored in the option table one line per user with the following structure :
USERNAME__MobileMessagingSettings = 
  array(
    'PhoneNumbers' => array(
                         '003354827548' => NULL, // NULL : phone number is activated
                         '00542857842754' => 52123, // !NULL : validation token value
                       ),
    'ProviderAuthToken' => 'f4e8za4al', // Only if Super User or if MobileMessagingSettings.DelegatedManagement is set to TRUE
  )
  • Whether the auth token is managed by the super admin or not will be defined in the option table with the following structure :
MobileMessagingSettings.DelegatedManagement = TRUE // True if users can manage their own SMS account
  • The code managing the SMS provider will follow the Service Gateway design pattern to keep an open door for multiple SMS providers.
Highlight significant increases/decreases in Reports (movers and shakers)
#57
  • Opened: October 31, 2012

    Modified: October 31, 2012

  • Priority: Major

    Component: New Plugin

  • Milestone: Milestone 2

    Owned by: User one

Most popular pages/keywords/etc. are popular every day, and it is hard to know what changed since the day before. What would be interesting would be to visualize:

  • Which pages significantly increased/decreased?
  • Which URLs links/SE keywords brought me significantly more/less traffic compared to last period?

This could help answer questions such as:

  • Why did the website traffic suddenly decrease? Traffic from some Keywords/Referers significantly decreased, and some pages significantly were less seen.
  • Similar with answering “Why did traffic suddenly increase?” Why are my Ecommerce orders growing so much, how do I know the overall positive contributors explaining our growth?

User interface updates for new reports & hints

New report: Overall Movers and Shakers

This report would have to query in particular: page URL, page Titles, Keywords, websites, campaigns, maybe 8 reports, for current AND previous periods. Making 16 API calls should work fine and comparison would be pretty fast hopefully.

This report could be available as a widget showing the top 5 movers and shakers overall (increased keywords, pages, etc.).

Extend existing report: Highlight Movers and Shakers

Currently reports display “Label”, “Visits count” by default. There is a button to display 5 more columns and a view for Goal conversions metrics.

We could maybe improve at least the “basic” view showing only visits.

In this view we could for example display in a new column on the right, a green or red % increase, when there is a a major change (threshold set to +/- 10% for example). Otherwise leave blank. See also #291

Below the report, or above?, we could display these rows from the report that have significantly changed since yesterday.

We could put threshold of minimum visits impacted = 20
minimum % increase or decrease = 2% for example?

So it would not display minor changes that are not interesting.

They could be displayed on top of the report

*Best countries*
(green sign) India (+50%) and Scotland (+33%)
(red sign) France (-33%) and Germany (-44%)
|---------|--------|
| Country | Visits |
|---------|--------|
| India   | 176    |
| Scotland| 54     |
| France  | 23     |
| Germany | 23     |
[....]
Next >

Display evolution % next to all sparklines

Idea is to show the increase and decrease of Visits, Conversions, etc. basically next to each Sparkline. There is a proposal mockup at: http://dev.piwik.org/trac/attachment/ticket/291/sparklines.png

Implementation

To produce a report “Period VS Previous period” for each keyword, we would need to select all keywords from both periods (2 datatables), and compare both to extract significant increases/decreases.

The code would be optimized so that it returns in less than 2 seconds when calling 8 different APIs so comparing 16 datatables.

The algorithm will typically extract:

  • items with a growth of N %
  • new items (new keyword since yesterday) with at least M %
  • Never show items with less than P visits
  • By default, we should enable the display in the UI for only the very significant changes, with a high threshold of visits impacted. The feature should not bring noise if the changes are not that interesting.

New widget to show this great insights:

  • New smart widget to show what’s changed in dashboard automatically based on last login date: #3250

Similar tickets

  • compare 2 periods: #291
  • show evolution for a given keyword/page: #534
Page Transitions (clickpath) to show previous/next pages for any selected page (ClickPath Page Analytics now available since Piwik 1.9!)
#3332
  • Opened: October 31, 2012

    Modified: October 31, 2012

  • Priority: Normal

  • Milestone: Milestone 2

    Owned by: User one

Awesome feature: Analytics Previous / next pages

For each page URL, this plugin reports the top X

  • previous pages
  • following pages
  • outlinks
  • downloads
  • campaigns
  • search engine keywords
  • external websites

The information is given in a popover which can be launched via a table action on the pages report.

Advanced internal site search tracking & analytics, for internal search engines and keywords (Site Search Analytics now available since Piwik 1.9!)
#2992
  • Opened: October 31, 2012

    Modified: October 31, 2012

  • Priority: Normal

    Component: Core

  • Milestone: Milestone 2

    Owned by: User one

The goal of this ticket is to describe the Internal Search Tracking & Reporting feature for Piwik core.

Currently, a third party built by Timo exists in #49.

We will reuse this great plugin as much as possible, integrating it into a Core plugin, maybe improving the features, user interface, documentation, and checking that it is fast & optimized on high traffic piwik servers.

More specifications and details to come later.

User Interface & more!

Graphs list of Improvements (jqplot+js)
#3110
  • Opened: December 15, 2012

    Modified: December 15, 2012

  • Priority: Normal

  • Owned by: JulienM

This ticket is a placeholder to link to all tickets and ideas related to improving the JS/Jqplot Graphs in Piwik.

See the ideas for improvements which all have their own tickets. We will not implement all of them before 2.0, but these are the best ideas and we shall implement most of them!

Help us fund Piwik 2.0. Contact us to donate for a particular feature!

  • #3109 Support horizontal bar charts for better data visualization
  • #3082 “Actions per visit” and others should be plotted on a new Y axis with lower values
  • #2829 Axis display percentage sign
  • #2786 Plotting any filtered /searched row
  • #2159 Allow to plot more than 30 last X, for example plot last 265 days of revenue or visits..
  • #1253 Annotations / markers displayed in the graph
  • #1915 Saving and restoring the picked metrics for each user
  • #3528 Display Goals metrics in the picker
  • #2955 If you try to view a graph that spans over a year or so, you get unreadable X axis labels
  • Below Evolution Graph, a new “Cog” icon feature “Show running average line” would show the running average for the current evolution graph (was suggested in #1718).
Replace Flash graphs with Javascript + Canvas graphs charts (now available in Piwik 1.5!)
#2431
  • Opened: November 1, 2012

    Modified: November 1, 2012

  • Priority: Critical

    Component: UI - UX (tpl, js, css)

  • Milestone: 1.5 - Piwik 1.5

    Owned by: EZdesign

Provide charts (pie, bar and evolution) rendered by JavaScript rather than Flash.

Suggested library: jqplot (http://www.jqplot.com)

  • Open source and free
  • Updated frequently
  • Highly customizable

Open questions

  • Do we add JS charts in addition to or instead of Flash?
Email reports changes for better UX improvements
#3183
  • Opened: November 1, 2012

    Modified: December 15, 2012

  • Priority: Normal

    Component: UI - UX (tpl, js, css)

  • Milestone: 1.9.x - Piwik 1.9.x

    Owned by: JulienM

After using Email reports with a few new Piwik users, I’d like to propose some minor yet extremely useful changes that will make the feature easier to use and more enjoyable:

  • New feature (could be in a different ticket): By default downloads/outlinks/pages reports could be “flattened” and truncated after X~23 rows
    • This feature could later be expanded (see #2137) but already it would be great to use with flattened!
  • Default to HTM type as it is the most portable and quick to read
  • std piwik Feedback on successful save/ edit/ delete / report sent
  • Make clear report that clicking “Send now” will send report for CURRENT select period in the calendar. Could we write in small letter the date “Send report now [below, in grey] Date: 26 Mai 2012″ ? Even better, could we display the date only when the link “Send now” is hovered?
  • The fact that a report is sent for a specific website should be more clear. Could the title on the manage report page say “Manage Email reports for website $name”
  • Create scheduled report link should be bigger esp. when there is no report
  • (optional) Display options -> should be “Customize report”
  • When “All websites” is clicked, the generated report should be slightly more clear:
    • the text “Website $name” at top would be moved to after the All Websites dashboard.
  • If Manage report for a Super User, there could be a new Link “Click here to upload a custom logo to display instead of the Piwik logo.”. THe link is displayed only if logo not already customised.
Make ‘Row Evolution’ Feature Even More Awesome
#3158
  • Opened: December 15, 2010

    Modified: December 15, 2012

  • Priority: Normal

  • Owned by: EZdesign

There are many items which I believe will make the feature easier to discover, easier to use, harder to misuse! here they are:
‘ Usability & UI New feature Review

Small UI changes

  • When Row evolution icon is hovered, display a darker grey icon highlighted
  • When looking at row evolution for a particular URL, it would be nice/consistent if the URL in the popover was linking to the actual URL (target=_blank) and showed the yellow icon
  • On hover on the icon, display a help text “Detailed report for %s”
  • Title of the popover: For pages, it could include the yellow logo for pages. The title should link to the URL in target=_blank.

Usability / experience

  • When user presses the SHIFT key, it would be beautiful to highlight with a pale yellow border (or sthg else), the other icons from the other rows in the current mouse-focused data table. The hilighted pale orange + icon will make it clear that many rows can be selected and compared together.
    • When a row is “Multi-clicked” the pale orange and darker icon stay visible for this row
  • Loading data… could be instead “Loading Evolution data for {here icon and label name}” this could be done by copying the HTML of Label row, including the icons/logos?
    • On the demo and my PC, it takes between 7 and 15 seconds to load a Row evolution report. Users will see this message a lot, so if it prepares the brain for the analysis it is good :)
  • Comparing Multiple Rows
    • when changing the metric: it would be nice to not remove the previous report, but display “Loading data…” at the bottom and leave the current metric report.
  • Text “Metrics for Page URL: photos.php” should also include yellow logo + link to the page URL in a new tab
    • Similarly Referers Websites should be linked to the actual URL in the title + “Metrics for Website: %s” text

Integrate Goals / Ecommerce in Row evolution
How could we allow to display metrics of “Goal Revenue, Ecommerce revenue” and then per Goal metrics: “Goal %s revenue”, “%s conversions”, etc.

  • Goals are a key report that users will want to know about historically. For example, “compare the ecommerce revenue for the top 4 search keywords or entry Pages over time”.
  • It would be useful to show by default the Goal metrics when &viewDataTable=tableGoals is currently displayed (the user is analyzing Goals so Row evolution could report “Visits + Page Views + all other Goal metrics”
    • The problem is we can’t fit all goal metrics in the row evolution report (for example if there are 5 goals, that’s already 10 metrics).
  • So we could display Visits + Pageviews + Revenue + Conversions + Conversion rate for all goals
  • A new metrics picker SELECT element would allow to refresh the report to display these metrics for a specific goal instead of the goal average
    • Row evolution will then allow to plot any metric of any row in Piwik!!

Less important

  • Row Evolution Footer: add a link to view the Visitor Log filtered for users who used this [search engine keyword], [browser], [time of day], etc. We would need to know which segment it refers to, and generate the &segment= URL value.
  • On the Evolution Graph, clicking on a different date should reload the same report (without Row evolution report) for the new date
  • The Row Evolution report could be widgetized: IFRAME include link for a specific report, useful for SEO reporting, emailing the report by email, etc.
  • When the popover finishes loading, it would be nice if the browser would scroll smoothly to the top of popover, the same way we scroll when sub datatables are loaded. This would be useful if user goes up and down in the 5-10s of loading row evolution.
Allow users to plot any metrics in graphs with a drop down (now available in Piwik 1.7!)
#1820
  • Opened: November 1, 2012

    Modified: November 1, 2012

  • Priority: Major

    Component: UI - UX (tpl, js, css)

  • Milestone: 1.7 Piwik 1.7

    Owned by: matt

It would be great for usability, reporting efficiency and consistency, that all graphs in Piwik have possibility to select which metrics to plot.

In addition to the current implementation, the following features would be nice:

  • Saving and restoring the picked metrics for each user
  • Comparing the conversions of different goals
  • Make cart metrics selectable in ecommerce reports. This is hard to do because ecommerce conversions are tracked with idgoal=ecommerceOrder, which means they are not in the main report but in the goals dimension.
Allow to “switch off the lights” to enjoy a simpler UI with less clutter
#3093
  • Opened: November 1, 2012

    Modified: November 1, 2012

  • Priority: Major

    Component: UI - UX (tpl, js, css)

  • Milestone: 1.9.x - Piwik 1.9.x

    Owned by: capedfuzz

The Piwik dashboard is very useful and practical, but it can be a daunting for a first time user.

One aspect I think is that there are too many pixels on the dashboard, too many blocks and lines to process.

I think a simple but cool feature could be:

  • Add a light bulb icon, in opacity 0.5 for example, next to “About Piwik box”
  • When clicking on this icon, we change a few CSS style to remove
    • Widget top bar grey background
    • add opacity 0.5 on datatable footer icons
    • remove widget 1px border
      • These changes help user focus on the actual data.

See screenshot for example on how it looks like. There could be more changes I’m sure.

  • When the dashboard is light off, the icon is different
    • Can be switched back on.

But maybe even better… Could we simplify the Piwik design in all cases (and not make it an option) ?

What do you think of the attached file?

Add a new column in all reports to report the percentage of visits
#1816
  • Opened: November 1, 2012

    Modified: November 1, 2012

  • Priority: Major

    Component: UI - UX (tpl, js, css)

  • Milestone: 1.9.x - Piwik 1.9.x

    Owned by: matt

From email “I have always missed the simple ability to view the stats in percentages (as a third column). Most of the time, the percentage (let’s say the percentage of mobile users as an example) is much more useful than the absolute numbers.”

This would be a simple yet very useful add to Piwik.
Some challenges are:

  • do we display % visits, % unique visitors or % pages? not obvious, even though % visits is the most commonly useful.
  • do we add a new column in all widgets (in dashboard), or only in the main reports (when clicking on the menu)? Adding a new column in all widgets is a ‘major’ presentation change, but maybe worth it.

Help us reach Piwik 2.0

The Piwik team are volunteers and need your support to continue working on Piwik and innovate with new features. We need your help to implement Piwik 2.0.

If you or your company would like to sponsor a particular feature from the list, contact us now to sponsor a feature.

If you have a smaller budget (starting at 30 euros but you can fund more), you can help fund a feature with our new Crowdfunding Platform: you choose the amount you donate and you get one or several cool Piwik perks in exchange... plus the amazing knowledge that you directly contribute to the development of your chosen project!

Build the Official Piwik Plugins Repository at plugins.piwik.org

With Piwik 2.0 on the way, we make an official call to the community, to start an open Marketplace of Analytics plugins for Piwik. Piwik is the leading open source analytics platform, and we believe that an open Plugins Repository will being in exciting new reports, functions, visualizations, and more. There are already more than 30 plugins listed in trac, but we will build a better platform to search, install in 2 clicks, rate, based on github. This work is a multi-tickets task, find out more: Piwik Plugin Repository Project.

No Bug Policy & Quality Assurance

We practise a "no bug" policy; this means we will usually give high priority to bugs so they can be fixed quickly. We like Piwik to 'just work' for everyone. All new bug reports are assigned to the current Milestone in our Bug Tracker software.

To prevent bugs and ensure stable software, Piwik extensively uses automated tests. A large number of tests run automatically after each modification to the source code. Having so many tests in Piwik code base helps us to be agile and release new features regularly, while ensuring that there are no problems with new functionality. Learn more about our Quality Assurance process and automated tests in Piwik

Security

We are constantly working hard to ensure that Piwik is a secure software that everyone can trust. Piwik is used on 460,000 websites of individuals, companies, big enterprise businesses, and many governements agencies. We already conducted two proactive professional security audits and are constantly working on ensuring high security standards. There has been hundreds of professionally trained security experts scrutinising our source code as part of our bug bounty program! Learn more about Security in Piwik.