Google Analytics Data Retention policy - which reports does it limit?
From 25th May 2018 Google allowed you to automatically wipe user-level data from the reporting from before a cut-off date, to better comply with GDPR. We made the change for Littledata's account to wipe user-level data after 26 months, and this is what we found when reporting before February 2016. Reports you can still view before the user data removal Audience metrics Pageviews ✓ Sessions ✓ Users X Bounce rate ✓ Audience dimensions Demographics X OS / Browser X Location X User Type X Behaviour Pageviews ✓ Custom events X
Six challenges in developing a Shopify integration
At the start of 2017 Littledata released its first Shopify app. A year on, here are my observations on the technical challenges we’ve overcome. This week we're at Shopify Unite in Toronto, and it's no surprise that their app ecosystem continues to grow. We chose Shopify as our first platform partner due to their open APIs, quality documentation and enthusiasm from other developers. Much of that has been as expected, but to help all of you looking to build your own Shopify app I’ll share some of our learnings on the hidden challenges. Littledata's Shopify app makes it a one-click process for stores to set up for Enhanced Ecommerce tracking in Google Analytics, and then get actionable insights based on the Google Analytics data. It has to hook into Shopify orders and products, as well and modify the store's theme and process ongoing transactions. 1. Handling re-installs gracefully The great advantage of Shopify’s app store over, say, Magento marketplace, is that any store admin can install and pay for an app with a couple of clicks. The disadvantage is that stores can be as quick to uninstall as install. Store admins may start, realise they don’t have permissions, time or energy to continue and roll back to try again later in the day. Since our app inserts a snippet into the store’s theme layout (see point two below), uninstalling removes the web-hooks we set up but does not remove the inserted snippet. When a store re-installs our app has to work out what state they were in when they uninstalled (audit, test mode or live), whether the script snippet is still there and what settings have been changed in the meantime. It took us a few months to get a handle on all the possible user flows, and we’ve found automated end-to-end tests to really speed up running through the different scenarios. In our Meteor framework we use Chimp [link] to run tests through Selenium on localhost and on our staging server. We've also found it essential to track our own stats of 'installs + activations' (including the date of first install and time to finally uninstall) rather than relying on the Shopify Partner stats of uninstalls and installs, which can hide the detail in between. 2. Working with script tags The other side-effect of making apps easy to install is that you can assume the early-adopter stores who will try your app already have lots of other installs. Shopify recommends using the Script Tag API to handle scripts linked to the app, so that when a store uninstalls your app it also removes any client-side scripts from the store. Unfortunately, in early tests we found the load latency to be unacceptably high: on some stores, only 50% of the page load events were getting fired before the user moved off the page. So plan B was add a snippet to the store theme, and then load this snippet at the top of the <body> element on all the layout templates. This has worked much more predictably, except when theme editors remove the snippet reference without enquiring what the Littledata app does (see our fifth challenge). 3. Charge activation vs authorisation Now a very simple gotcha. In our first month we had around 60 installs at a flat price of $20/month, but apparently no revenue. After investigation we found we had not activated the recurring charges after the store admin had authorised them. Doh! We're still not sure why an app would want to have authorised charges which are not activated -- seems like over-engineering on Shopify's side -- but luckily it was easy to correct without asking for more user permissions. 4. Tracking adds-to-cart The first version of our app tried to run the script when customers got to the ‘/cart’ page of a store. The problem here is that many stores have AJAX or ‘mini’ carts where customers can checkout without every visiting the cart page. We looked to trigger the script before the user got to the cart the page, but this appeared to run too many risks of interfering with the customer actually adding the item. Our final solution has been to poll the Shopify store for the current cart, and see if products have been added (or removed) since we last polled (and stored the previous cart contents in memory). This is somewhat inefficient, as it requires continuous network activity to grab the cart JSON from Shopify, but we’ve reduced the network requests to one every 4 seconds – judging that customers are very unlikely to add a product and checkout in less than 4 seconds. This cart polling has proved more reliable across different store templates. 5. Integrating with other Shopify apps I mentioned that early-adopter stores tend to have lots of other apps: and those apps have loyal customers who push to make Littledata's app to work their chosen app (not just vanilla Shopify). The challenge is that most of these app development companies run a very Agile process, constantly changing how their app works (hopefully to improve the experience for store owners). An integration that worked a few months ago may no longer work. We've found the best solution to be open developer-to-developer communications, via a Slack guest channel. Having the developers implementing the features on each side talk to each other really cuts down the delays caused by a well-meaning project manager slightly misinterpreting the requirement. 6. Handling ongoing updates As tested improved client-side tracking scripts, we needed to update the script in the store theme (see point 2 above). This creates a small risk for the store, as there is no UAT or test environment for most stores to check before going live with the new script. The store theme may also get edited, creating new layout templates where the Littledata snippet is not loaded. In the first version of our app we tried to update and re-insert the latest Littledata snippet automatically on a weekly cycle. However, once we reached hundreds of active installs this became unmanageable and also opaque for the store admins. In the latest version we now allow store admins to UPGRADE to the latest script, and then we check all the correct Google Analytics events are being fired afterwards. Giving the end user control of updates seems a better way of maintaining trust in our brand and also removing risk: if the update goes wrong, it’s quicker for us to alert the store owner on how to fix. Conclusion I’m still sure we made the right choice with Shopify as a platform, as their APIs, partner support and commercial traction are all number one in the ecommerce world. But I hope that by sharing some of the hidden challenges in developing Shopify integrations, we can all build better apps for the community. Have you built something for the Shopify app store? Are there development problems you’ve encountered which I haven’t shared here? PS. Are you a developer interested in joining an innovative analytics company? We're hiring in multiple locations!
How Littledata helps Shopify stores comply with GDPR
When the GDPR regulation comes into effect later this month, it will impact all websites trading with EU citizens. That means any ecommerce site with customers in Europe! Is your Shopify store ready to comply? We recently updated our Shopify app (since release 7.8) to help Shopify stores which use Google Analytics comply with GDPR. In addition to automatic fixes to help your store comply, we include recommendations for how to update your site content (such as Terms and Conditions), and how to deal with the new 'two year rule'. If you're running a Shopify store, the time to act is now. Automatic fixes with our Shopify app The first two steps are done automatically when you install our GDPR-ready Shopify app. If you're already using Littledata's Shopify app, these two fixes can be applied when you upgrade to our latest tracking script (version 3.2). Here's what they address. 1. Anonymise customer IP addresses The IP address of your website visitor is considered personal information under GDPR, and to remove any risk that this is sent to Google’s servers in the USA, our script scrambles the last few digits of the IP address. Google already promises not to store the IP address, so this step is an extra level of safety. This slightly reduces the accuracy of tracking which city your visitor came from -- but we believe that this is a small price to pay for ensuring anonymity. 2. Filter personal emails and ZIP/postcodes from pageviews Many sites accidentally send personal data in the page URLs or titles tracked by Google Analytics. For example, apps with their own checkout often send the user email as a URL parameter like ‘/email@example.com’. Our script now filters that personal data out at source, so the page path you’ll see in Google Analytics is ‘/url?email=REMOVED’. Additional manual steps There are two additional manual steps to ensure that Google Analytics for your Shopify store is GDPR-compliant. 3. Update your terms and conditions You need to update your website T&Cs to ensure users are aware of the Google Analytics Advertising Features that our Shopify app activates and Google uses to identify user demographics, such as gender and interests. We are not lawyers, but we suggest using something similar to these sentences to describe what data is collected, how you (and we) use the data, and how how users can opt out: Our site uses Google Analytics Advertising Features to deduce your gender, age group and interests based on other types of websites you have visited. We use this in aggregate to understand which demographics engage with areas of our website. You can opt out with Google's browser add-on. 4. Remove user-specific information after 2 years You should also change the data retention period for your Google Analytics web property, so that Google removes all user-specific information from their database after 2 years. To make this change, logging to your GA account and go to the Settings cog, and then Property > Tracking info > Data Retention. Use the 'data retention' drop-down menu to select to keep user data for 26 months, and mark 'reset on new activity' to ON. This means that after 26 months, if the user has not come back to your website, any user cookie will be deleted. We think this sensible to comply with the Right to Erasure without making any practical limits to your analysis. Right to Erasure feature coming soon! We're also working on a feature to help websites comply with the Right to Erasure or Right to be Forgotten. Here's a summary of that aspect of the regulation, from the summary of key changes at EUGDPR.org. Right to be Forgotten Also known as Data Erasure, the right to be forgotten entitles the data subject to have the data controller erase his/her personal data, cease further dissemination of the data, and potentially have third parties halt processing of the data. The conditions for erasure, as outlined in article 17, include the data no longer being relevant to original purposes for processing, or a data subject's withdrawing consent. It should also be noted that this right requires controllers to compare the subjects' rights to "the public interest in the availability of the data" when considering such requests. Littledata's Right to Erasure feature will ensure that when you delete a customer from your Shopify admin interface, any references to that customer are deleted from Google Analytics. This won’t affect aggregate reporting, such as number of web sessions or transactions. When do GDPR regulations take effect? The official enforcement date for General Data Protection Regulation (GDPR) is 25 May 2018. At that time any organisations in non-compliance may face heavy fines. In short, we recommend implementing the fixes above ASAP for your Shopify store. All you need is Google Analytics account and our Shopify app. And do check our blog regularly for updates. This is the best place to hear about new Littledata features relating to GDPR, as well as news and analysis about how the regulations affect different types of online businesses, including ecommerce websites, subscription businesses, and membership-based sites such as large charities and nonprofits. Looking for additional support? Contact us about GDPR consulting for analytics setup.
Tracking the online customer journey for luxury ecommerce
Today I'm excited to be participating in the Innovation Meets Fashion event in Lugano, Switzerland. As an increasing amount of luxury and fashion retail moves online, high-end brands are finding it complicated to track the complete customer journey. In many cases, difficulties in tracking customers through to eventual purchase are holding back investment in the digital experience and online marketing. But it doesn't have to be this way. We've found a straightforward correlation in ecommerce between the average ticket price of the item being purchased and the number of web pages or sessions before that purchase is made. Simply put, customers spend longer considering big ticket items than they do with smaller ticket items and impulse purchases. Luxury retail involves many touch points with the brand across your websites, social sites and physical stores. The problem is that the longer than online customer journey, the harder it is to get consistent data on which top-of-funnel experiences are leading to purchasing. So first the bad news: since many potential customers browse anonymously, perfect ecommerce tracking across a long online and offline journey is not possible. Tracking browsers based on first-party cookies (such as Google Analytics) will fail when customers use multiple devices, clear their cookies or browse in-app (such as from Facebook). Yet there are three ways we have seen retailers selling high value items increase the reliability of their online behavioural data. 1. Track online shopping behaviour in detail Understanding whether customers browse certain products, view the detail of product variants and even add-to-cart is a good proxy for seeing which campaigns eventually convert. Does your brand have a good understanding of how each marketing channel influences browsing behaviour, after the landing page but before the checkout? 2. Offer a good reason to get customers to login before buying VIP offers, registering for events and discounts all offer a good way of getting customers to login from different devices. With the correct analytics setup, this login information can be used (without infringing the users’ privacy) to link together different interactions they make across multiple devices 3. Make the most of your email list Even without having a login before purchase, customers clicking through links in a marketing email can allow the same stitching together of sessions. This means that if a customer visits a link from their mobile device, and on another week from their home laptop, these two devices can be linked as belonging to the same email – and therefore the same person. Luxury online retail involves a complex journey. Littledata is here to make your tracking and reporting both easy and accurate. Sign up today to get started with our complete analytics suite, and feel free to reach out to our Google Analytics consultants with questions about best practices for luxury ecommerce. Your success is our success!
GDPR compliance for ecommerce businesses
Ecommerce companies typically store lots of personally identifiable information (PII), so how can you make compliance easier without compromising analysis? With the deadline for GDPR compliance looming, I wanted to expand on my previous article on GDPR and Google Analytics to focus on ecommerce. Firstly, who does this apply to? GDPR is European Union legislation that applies to any company trading in Europe: so if you sell online and deliver to European Union member countries, the regulations apply to you. It's essential that you understand how your online business is collecting and storing PII. Splitting PII from anonymous data points Your goal should be to maintain two separate data stores: one that contains customer details, from where you can look up what a specific customer bought, and one that contains anonymous data points, from where you can see performance and trends. The data store for the customer details will typically be your ecommerce back-end and/or CRM (see below). This will include name, email, address, purchase history, etc. It will link those with a customer number and orders numbers. If a customer wants the right of access all the relevant details should be in this store. We use Google Analytics as the anonymous data store (although you may have a different ecommerce analytics platform). There you can store data which only refers to the customer record. These are called pseudo-anonymous data points under GDPR: they are only identifiable to a customer if you can link the customer number or order number back to your ecommerce back-end. Pseudo-anonymous data points you can safely send to Google Analytics include: Order number / transaction ID Order value / transaction amount Tax & shipping Product names and quantities Customer number Hashed email address (possibly a more flexible to link back to the customer record) If a customer exercises their right to removal, removing them from the ecommerce back-end will be sufficient. You do not also have to remove them from your Google Analytics, since the order number and customer number now have nothing to refer to. You do still need due process to ensure access to Google Analytics is limited, as in extreme circumstances a combination of dimensions such as products, country / city and browser, could identify the customer. Isn’t it simpler to just have one store? Every extra data store you maintain increases the risk of data breaches and complexity of compliance – so why not just analyse a single customer data store? I can think of three reasons not to do so: Marketing agencies (and other third parties) need access to the ecommerce conversion data, but not the underlying customer data Removing a customer’s order history on request would impact your historic revenue and purchase volumes – not desirable Your CRM / ecommerce platform is not built for large scale analysis: it may lack the tools, speed and integrations needed to get meaningful insights Beware of accidental transfers There are a few danger areas where you may inadvertently be sending PII data to Google Analytics: Customer emails captured in a signup event A customised product name – e.g. ‘engraving for Edward Upton’ Address or name captured in a custom dimension Our PII audit check is a quick, free way to make sure that’s not happening. Multiple stores of customer details GDPR compliance becomes difficult when your customer record is fragmented across multiple data stores. For example, you may have product and order information in your ecommerce database, with further customer contact details in a CRM. The simplest advice is to set up automatic two-way integrations between the data stores, so updating the CRM updates the ecommerce platform and visa-versa. Removing customer records from one system should remove them from the other. If that’s not possible, then you need clear processes to update both systems when customer details change, so you can comply with the right to rectification. Conclusion GDPR compliance need not require changing analytics tools or databases, just a clear process for separating out personally identifiable information – and training for the staff involved in handing that data. I hope this brief overview has been helpful. For further advice on how your ecommerce systems comply, please contact us for a free consultation. Littledata has experience with every major analytics platform and a wide range of custom setups. However, as a number of global companies are concurrently prepping for compliance, we highly recommend that you get in touch sooner rather than later!
The 5 worst arguments for boosting Bitcoin
I’m exasperated reading dodgy logic justifying the heady ascent of Bitcoin. What are the worst 5 arguments I’ve heard? Full disclosure: I don’t own any Bitcoin, or have any bets on its rise or otherwise. 1. Bitcoin is an insurance against the collapse of capitalism The booster The rise of artificial intelligence and mass joblessness will sweep away much of the old order of nation states and their currencies. Bitcoin is independent of government and will survive the coming storm. A grain of truth I believe big change in the relative value of labour and capital, and how they contribute to the tax base, is coming faster than politicians expect. And the reactionary backlash in affected countries, such as those voting for Donald Trump, won’t stop this trend. The sceptic Bitcoin relies on a chain of other technologies which may well get disrupted with the collapse of capitalism: cheap power supply, a global internet and secure online vaults to hold the private keys and transact the Bitcoin. If you’re betting on the end of the world as we know it, hunting and farming skills are going to be more useful! 2. Bitcoin’s limited supply makes it deflationary by default The booster Unlike fiat money (e.g. the US dollar) which can be printed at will by central banks, the total number of Bitcoin is mathematically limited to 21 million. That means, as other currencies inflate, Bitcoin will hold its value – i.e. it’s digital gold A grain of truth As developed countries around the world are forced to borrow themselves out of the hole of shrinking tax bases and increasing healthcare costs, they may try to inflate their currencies to erode the debt. The sceptic Central banks have a positive inflation target for a reason: in a deflationary currency, no-one wants to spend the currency and so there’s no circulation of wealth. If one Bitcoin could have bought me a coffee in 2016, but at the time of writing could have bought a car, why would I ever spend it? And if no one spends the currency then it has no tangible value. 3. Bitcoin is the leader of the blockchain revolution The booster Blockchain is one of the few game-changing technologies to be invented the last two decades. It will revolutionise the world of finance, and you need to own Bitcoin to be part of that. A grain of truth The blockchain ledger, keeping a public record of all transactions, and reducing the possibility for fraud or interception, will certainly change many aspects of finance. There are many projects underway in financial trading and government. The sceptic Just because Bitcoin was the first use-case of the technology, does not make it essential to newer blockchains. Equally, its first-mover advantage may not even make it the winning cryptocurrency. That said, I wouldn’t go out buying a basket of other cryptocurrencies just yet – they are all overinflated by Bitcoin’s rise. 4. The increasing mining cost of Bitcoin underpins its value The booster New bitcoin gets exponentially harder to mine, so since the cost of electricity for the miner’s servers won’t fall, the cost per bitcoin mine is rising all the time. And if you can’t mine them, you’ll have to buy them. The sceptic Yes.. but what if no one needs Bitcoins at all? Mining gold is subject to the same economic forces, but if the gold goes out of fashion as a value store (as it did an the turn of the Millennium) it still had industrial value for conducting electricity and aesthetic value for jewellery. Bitcoin has neither of those. 5. The rise of bitcoin is 2017 shows it has won out as the cryptocurrency of choice The argument Bitcoin is now the established alternative store of value, which is why it has risen so fast in 2017. And what if all the pension funds and institutional investors now buy up a slice to ensure an allocation of this new asset class? A grain of truth There’s no rational way to value Bitcoin: it does not pay dividends or have intrinsic worth (see point 4). So it could be worth anything .. or nothing. The sceptic Every decade a new mania comes along for investors to follow. The vast chatter on LinkedIn, Facebook and other forums only heightens the mania by allowing unchecked falsehoods to flourish. You only have to look at the South Sea Bubble and Tulip mania to see there is nothing new under the sun. Enjoy the roller-coaster ride up .. because everything that goes up, must come down.
Retailers traded 2.4 times normal volumes during Black Friday week 2017
The results are in, and this year's Black Friday sales prove that things are continuing to look up for ecommerce. Across 570 online stores, the average store did 2.4 times their normal sales in Black Friday week 2017, compared with only 2.2 times in 2016 – and a greater proportion of stores participated in the sales. Following our post on pre-Black Friday trends, Littledata looked again at what happened from Thanksgiving Thursday 2017 through to the following Wednesday (the week including Black Friday and Cyber Monday) – versus a control period of November & December in 2016. Compared with 2016, we found a bigger number of stores participating in Black Friday sales this year: 53% of stores were trading more than 1.5 times their normal volumes, compared with only 49% in the equivalent week in 2016. For those stores which promoted heavily in 2016, the median boost was 2.5 times normal. And those in the bottom quartile of sales in 2016 still traded 108% their normal volumes. How did Black Friday promotions work for your store? Use our industry benchmarks to find out how your online store is performing against the competition.
Black Friday discounting increases next season’s purchasing
I knew Black Friday had reached ‘late adopter’ stage this week when a company I’d bought fencing panels from - fencing panels – emailed me their holiday season promotions. But the real question is whether all these promotions serve to drive customer loyalty or just attract bargain hunters? At Littledata we looked at aggregate data from 143 retailers who participated most in 2016 Black Friday, versus 143 retailers who did not. For the first 23 days of November 2017 – before Black Friday – the median year-on-year increase in sales was 13% for those pushing discounts the previous year, versus only 1% growth for those avoiding Black Friday discounting *. Our conclusion is that retailers who discounted most heavily on Black Friday 2016 saw a lasting benefit in extra sales a year after the sales period. However, we don’t know whether these extra sales were profitable enough to pay for the seasonal promotions. Another possible explanation is that higher-growth retailers are more active in marketing Black Friday, but in either event the discount season has done them no harm over the following year. In a follow up post next week we’ll compare the peak discount trading – and see if on average these same stores increased their participation this year or reigned it back. Looking at 2016, it seems Black Friday was bigger than the year before for our cohort of 270 UK retailers – but at the expense of sales later in the season. Yet in the UK we are not close to US-levels of hysteria yet, where a much greater proportion of the last quarter’s sales are done on that weekend. The other interesting question is what sectors does Black Friday affect? Reflecting back on my 2016 post, it may be a surprise that the biggest boost of over 100% average increase in sales comes for Health & Beauty stores; whereas technology and computer stores on average saw a boost of 40% for the week. (The graph shows the difference with the average sales volumes in November & December, by sector, for 3 selected weeks.) And perhaps I shouldn’t have been surprised by those fencing panels: business and industrial sites saw a big boost too! Interested in tracking online sales activity for your own site this holiday shopping season? Littledata's ecommerce analytics software provides accurate data and automated reporting to help you track promotions and drive conversions and customer loyalty. * The statistical detail I took a group of 573 retailers we have tracked for at least 2 years, and looked at the ratio of Black Friday weekend sales (Friday, Saturday, Sunday, Monday) to the 2 month average for November and December. Those in the top quartile (trading 2.6 times above average during the Black Friday season) were deemed to have participated; those in the bottom quartile, showing a dip in trading over that weekend were deemed not to have participated. I then looked at the year-on-year growth in revenue between November 2016 (first 23 days) and the same period in November 2017, for the discount versus non-discount group. A t-test between the groups found a 18% probability that the two groups had the same mean, not allowing us to dismiss the null hypothesis.
Subscribe to our blog
Get the latest posts in your email
Get the app
See for yourself why Littledata is the smartest ecommerce analytics appFree trial