Whether we appreciate it or not, everything we do in our daily lives is measurable by some metric. Therefore, data could be collected on every aspect of our daily lives. The advent of modern technology in day-to-day life has meant that there has been a mammoth increase in the amount of this kind of data being generated. In many cases, this kind of data has made huge changes in the way we think about and live our lives. Not only has this made our lives better but has allowed for major advancements in areas such as human behavior, conservationism, urban design, and public transport.

Therefore, I thought it was rather fitting in the spirit of this to use data to really explore some aspects of my life. I set myself two small challenges:

1. The first of which was to explore some of the data that was accessible to me that has been collected through my interactions with technology. All made possible thanks to Article 15 of GDPR, which gives individuals the right to request a copy of any of their personal data which are being ‘processed'.

2. Secondly, to perform some very basic analysis of this data to find some fun, interesting, or useful insights into my life.

The first of these missions was to find just some of the data that has already been generated for me. Very quickly I realised it was apparent that there were reams of different points that I could request access to and download, such as:

• Facebook Data – including posts or photos that I have shared, photos I am tagged in, posts I have liked, and groups I belong to amongst a whole host of other things.
• Instagram Data – which posts I have liked or commented on, who I have followed and who follows me, and what messages I have sent.
• Google data – my Google advertising profile (allowing a peek into how I am seen online), locations that I have visited (the first of which I could find was from 2011), and a full log of all the YouTube videos I have watched.
• Spotify data – all my listening habits, how long I spent listening to various tracks, at what time, and on what day.
• Apple Health Data – Steps and distance walked or run (from mobile phone sensors), any connected Apple Watch data (BPM, temperature, etc.), and even mobility data about how ‘well' I walk.

This is not an exhaustive list just possibly some of the platforms that are most widely used across the globe. The number of different data sources is likely to be endless and will only grow with time. However, for these purposes, I decided to look at what I could find out about my life from Spotify and Apple Health data.

The Highlights

Since the 10th of November 2015 I have walked at least 15,340km or 20,577,340 steps. That's roughly equivalent to walking from Cardiff to Queensland, Australia.

Climbed 44,471 flights of stairs, according to Apple Health this is one flight is worth 10ft. So aggregated (according to Apple's website) this is 135km or the same as 161 and a half Burj Khalifa's.

In the time period 13th February 2021 to 13th February 2022, I have listened to 55, 537 minutes of music or audio on Spotify. Compared to the Spotify annual reported amounts of 42,875 minutes and 25,6440 minutes in the calendar years 2021 and 2020. A YoY increase of 67.2% and 29.53% respectively.

On average listened to Spotify the most on a Thursday and listened to the most songs in February 2021.

Some other quick stats:

• Most songs on average were listened to at 19:00
• Average number of songs listened to daily – 196
• Listened to 2609 different artists
• Listened to 5322 different tracks

Highlighted Graphs

Fig 1. Bar Chart showing the number of steps I've taken per year. Data from www.trovalo.io

Fig 2. Total number of songs listened to on Spotify by day of the week.. Data from www.trovalo.io

Fig 3. Maximum number of songs on a day

However, the purpose of this article was never to highlight the amount of data that big companies have collected on me or the ways in which they could process it to understand more about me. Instead, highlight the number of different ways that the same dataset can be represented to understand daily events and processes and draw attention to some inventive ways individuals and companies can use data to enhance our daily lives and progress society. Nobody is suggesting that this data analysis is one of those examples or that it has revealed much in the way of meaningful insight.

It was all made possible thanks to legislation allowing individuals to have greater control over their own personal data as highlighted at the start of this article. I believe that legislation just like that will lead to an even greater data-sharing economy.

One that incentivizes all participants to take part. Here are some examples:

• Monetisation over a current existing asset that is not utilised in this way.
• Better life outcomes and general day-to-day life
• Advancements in Science and Technology

Whilst also balancing the privacy rights of those involved with this data collection, by adhering to the following:

• Total anonymisation of data but instead different aggregations of this data to show trends.
• Ensuring robust licensing systems are in place to ensure that data is only used for the purposes it was collected for and proper access controls.
• Ensuring no bad actors obtain personally identifiable data for sinister purposes.

I would invite you to think about some of the ways personal data from your own lives could be used to make better decision-making within the context of your own life. This was a super simple process so if you want to try this for yourself I've added some YouTube tutorials on how to download and request your personal data from third-party providers.

The post Collecting Apple and Spotify Data on Myself : Here is What I Learned appeared first on Datafloq.

The post Shift Dev appeared first on Datafloq.

That very well may change. In late November 2017, the richest company in the world gave Quartz a glimpse of how its deep neural network, VoxelNet, identifies data points from a LiDAR sensor. This is part of Apple's effort to develop AI for a self-driving car. Originally, it looked like there would eventually be an Apple iCar out driving around on its own cognizance. But the company has since scrapped efforts to produce a car in favor of concentrating on software for what Tim Cook calls the mother of all AI projects. To have the best self-driving car, you have to have the best AI.

Will Apple be able to compete in the battle for the best AI, the mother of all tech battles? If the iPhone X's features are any indication, the answer is yes. Face ID uses a TrueDepth camera to analyze 30,000 invisible dots, creating a precise depth map of your face. Then, the neural network stores your facial data and compares subsequent datasets to the original, applying deep learning in order to adjust to changes in your appearance over time. This is the first application of face ID in a smartphone.

While Apple has been largely silent on its work with AI, companies like Google, Facebook, and Microsoft have published plenty of papers in academic journals. This would give anyone the impression that these companies are more intent on making real strides with AI, while Apple appeared to be content with making incremental progress on voice recognition and Siri.

But then, Apple published its first research paper late in 2016, placing it squarely within the ranks of every company working on solving the problem of how to make a machine generate new data from old input. Possibly, the paper and Apple's latest revelation about its self-driving car software place the company ahead of the pack. The paper, titled Learning from Simulated and Unsupervised Images through Adversarial Training, won one of two awards for best paper at the Computer Vision and Pattern Recognition conference. The other winner was Facebook.

Apple's research is admirable. Unlike Facebook and Google, Apple doesn't harvest raw user data to train its AI. A great deal of data is necessary to get a machine to learn ‘to get a neural network to create a new algorithm, which initiates a new process, based on analytics. When it comes to something like image recognition, Facebook and Google both have huge troves of user data they feed into their networks. When Facebook tags your picture without a human telling the network who you are, you might not be aware that old photos of you helped train the software how to do this.

Not so with Apple. The company's new machine learning blog details its process, which involves local differential privacy: Local differential privacy has the advantage that the data is randomized before being sent from the device, so the server never sees or receives raw data.

Our system is designed to be opt-in and transparent. No data is recorded or transmitted before the user explicitly chooses to report usage information.

The research team has trained VoxelNet by feeding it real images and then instructing it to alter synthetic images to make them look more like the real thing. After that, the network compares the altered synthetic image to the real image and makes a judgment as to which one is real. It then applies this judgment towards identification of new images.

The Verge provides a little more detail on how VoxelNet works. The software takes LiDAR data points, which are 3D pixels called voxels, categorizes them and compresses them in the neural network in order to create a single, active map of a car's surroundings. According to Roland Meertens, a Dutch engineer who builds computer vision systems for autonomous vehicles, this isn't so much a breakthrough for self-driving cars as it is an impressive application of LiDAR data for other AI purposes. The Verge reports that researchers Yin Zhou and Oncel Tuzel benchmarked VoxelNet's performance against a number of rival programs, and it handily outperformed them.

This indicates Apple is further along with AI than anyone thought. But in order to beat frontrunners like Waymo and Tesla, Apple will have to make some spectacular moves. The iPhone X has been labeled the best smartphone you can buy, a breakthrough unrivaled by anything else in the industry. It's not hard to imagine Apple rising to the occasion with its AI too.

The post Will Apple Catch Up to Its Competitors in Artificial Intelligence? appeared first on Datafloq.

The potential behind AR is difficult to underestimate. A report by Digi-Capital states that the augmented and virtual reality markets are likely to reach a total of $108 billion by 2021 with mobile AR‘s share being $83 billion compared to VR's share of $25 billion. Let those number sinks in…

ARKit is expected to help Apple take the majority of the mobile AR apps market's stake. Yet, most people know very little about Apple's AR framework and have no idea what new opportunities it provides.

So, let's find out!

What is Apple's ARKit?

First of all, you should understand what ARKit is and which tools you will need to start using it. ARKit is a framework that allows you to integrate sensors (a camera, accelerometer, and gyroscope) with your mobile application to add 2D or 3D elements to your iPhone's or iPad's live view.

In other words, ARKit helps you blend virtual objects or information with the real environment around you. To start using ARKit, you need two components:

• Xcode 9 editor. It's the latest version of Apple's code editor which allows you to create applications for all Apple platforms. Be it an app for macOS, iOS, watchOS, or tvOS, Xcode is a universal tool for developers. Xcode 9 contains the iOS 11 SDK, which means you can't use ARKit if you have an earlier version of the editor.

• An iPhone or iPad with iOS 11. ARKit is supported by iOS 11 only, so it's impossible to use Apple's AR framework if a device runs on an earlier iOS version.

What Opportunities Does ARKit Provide?

Now that you know essential information about Apple's augmented reality framework, you might be asking whether there's anything special about it. In fact, there are multiple other AR SDKs that can help developers create augmented reality experiences.

So, let's find out what unique opportunities Apple's ARKit provides.

It Develops Apps for Millions of Potential Users

As we've already mentioned, iOS 11 contains ARKit which means millions of people can install mobile AR apps on their Apple devices. That's a huge market with a gigantic target audience!

It's important to note, however, that iOS 11 isn't the only prerequisite for ARKit, as the framework runs only on A9, A10, and A11 (the latest) processors in Apple devices. So, if a gadget runs on iOS 11 and has one of these chips, it will definitely support mobile AR apps created with ARKit.

Here's a list of Apple devices that support ARKit:

Ensure Your AR Apps' Smooth Performance

Since ARKit is built into iOS 11, developers can create mobile augmented reality applications that run smoothly as there's no need to add a layer for image processing. That's an advantage of apps created with ARKit versus similar applications built with other AR SDKs.

High performance is particularly important for augmented reality apps as users need to have a feeling of reality.

Create Realistic AR Experiences

Would you like to use an app with poor graphics? That is not likely.

That's why developers should use the most technologically advanced tools to build augmented reality applications. ARKit has many features to be proud of, so let's mention the most important ones.

• Firstly, ARKit uses an advanced technology called Visual Inertial Odometry (VIO) to track the surrounding environment. VIO uses the iPhone's (or iPad's) camera and motion sensors, so Apple devices can accurately sense how they move and render virtual objects accordingly.

• Next, ARKit can analyse the scene and detect horizontal planes (such as floors or tables). Once a plane is found, virtual objects are placed on it.

• Finally, ARKit ensures a high-quality lighting adjustment. Apple's AR framework uses the camera to estimate the amount of light and adjusts the lighting of virtual objects.

Thanks to these technologies, Apple's AR framework allows developers to create realistic augmented reality apps that truly engage users.

Build Top-Quality Graphics with Third-Party Tools

Apart from offering a number of cool technologies, ARKit allows you to go the extra mile and use the power of third-party tools to deliver the highest-quality graphics in your mobile AR apps.

• Unity. The Unity ARKit plugin gives access to ARKit's features (we've mentioned them in the previous section) as well as additional Unity functionalities.

• Unreal Engine. The augmented reality plugin for Unreal Engine 4 allows you to apply the potential of one of the world's leading game engines to your mobile AR apps.

Add Face Tracking to Your AR Apps

So far, this functionality is only available on the iPhone X, but it's a revolutionary innovation in the AR industry. As you probably, know, the iPhone X boasts accurate and real-time face tracking the sensors on the iPhone X can detect the position and expression of a user's face.

ARKit can use this functionality to create unparalleled AR experiences. You can build stunning mobile AR apps and apply face tracking to virtual objects. So far, iPhone X owners can create and share custom animated emojis (also called animojis), but face tracking with ARKit allows you to build your own unique experiences with mobile AR apps.

Conclusion

Now that Apple has officially released iOS 11 and ARKit, you can start using this framework to create exciting mobile AR applications. ARKit is easy-to-use and you can try building your first AR app by checking out this step-by-step guide.

The post New Opportunities in AR with Apple’s ARKit appeared first on Datafloq.

Not long ago, my attitude was who gives a f#@% about the Fitbit? Consider that attitude changed. The Fitbit and other health-conscious wearables present the chance for the medical community to make a real difference through big data.

First, the data on wearables:

• 86 percent of health and wellness providers believe wearables and mobile apps will increase their knowledge of patient conditions
• 76 percent feel wearables will help patients with chronic diseases
• In the first quarter of 2017, the wearables market grew by 18 percent, an increase of 24.7 million devices; International Data Corp. (IDC) believes this is just a fraction of what's to come

T-Mobile bills this as a wearable tech revolution, one in which you can stay connected and track your life every step of the way. Along with Fitbit, the biggest names in tech are vying for a piece of the wearables market, including Apple, Samsung, and Google. That's because IDC's prediction on the tech's pending popularity are a good bet. One reason why it's a good bet is that the medical community has a growing relationship with wearables and the data they record.

Medical Research and Big Data

Health informatics professionals are excited about wearables because of the opportunity for crowdsourced medical research. Apple's ResearchKit open source framework is one promising example. The framework allows health organizations to create apps that work with the iPhone and Apple Watch.

Some of the apps that organizations have developed with ResearchKit only require an iPhone. Others work in tandem with the Apple Watch. Through the Apple Watch, users can transmit health-related data to apps such as SleepHealth and EpiWatch.

EpiWatch collects data on the onset and duration of seizures for people diagnosed with epilepsy. Doctors can then compare data with medication regimen to get an accurate picture what's working and what isn't. Over time, this will give researchers a better idea of which medications are most effective for patients with differing genes and differing lifestyles. What's more, Apple Watch sends an alert to the patient's family or caregiver when a seizure is starting.

Like the name suggests, SleepHealth is helping researchers gather data to help people sleep better. The Apple Watch monitors how alert the user is during the day, and then the user reports their nightly sleep patterns and quality. The app's intent is to figure out how conditions such as diabetes and obesity play a role in sleep health.

Through ResearchKit, users give medical institutions legally-valid informed consent to use their medical data for research. That's one of the reasons why ResearchKit is so promising: it includes a framework that works well with our current patient confidentiality laws.

Personalized Healthcare and Big Data

According to economist Peter Orszag, the US spends up to $700 billion a year on medical treatments that don't help patients. Personalization promises to streamline treatment, so patients get the care they need and billions of dollars aren't wasted. Wearables can transmit the data that practitioners need to make each diagnosis and each prescription accurate and relevant for the individual patient.

Personalization is a major breakthrough in healthcare. But according to researchers from South Korea and Tasmania, when it comes to wearables and personalization, There is a lack of techniques and approaches which exploit the sensory data collected by wearable devices and use them for decision making and recommendations.

In other words, there's no comprehensive and effective analytics infrastructure in place. Wearables ‘including watches, heart monitors, clothing, and sleep sensors ‘can transmit a great deal of data at an alarming rate. It's up to medical organizations and government to use analytics to do the following:

• Filter valuable data and discard irrelevant noise
• Look at the individual's medical information and make conclusions about efficacy of treatment
• Use data to make predictions and prescriptions for the patient
• Anonymize data and put it in a database
• Use database to optimize treatment for people diagnosed with conditions that researchers have been able to treat successfully by using data from wearables and smartphone apps

It sounds complicated because it is. But that doesn't mean it can't work. Companies are already using the internet, apps, and analytics to personalize marketing efforts. If government and healthcare organizations can navigate through the maze of our system and focus on the goal of personalizing healthcare for everyone, there's no reason why wearables and big data shouldn't be able to revolutionize patient care.

The post How Wearables Could Revolutionize Healthcare Through Big Data appeared first on Datafloq.

Pandora Now Offers Audience Data to Musicians

Following almost a full year after Spotifys very same announcement, Pandora announced Oct. 22 that the platform will open its analytics backend to musicians, giving them unparalleled access to data on their audiences music preferences. The data will include information on each song (including the total number of plays and the number of thumbs up), as well as the number of fans who have created a station for each artist, plus geographic and demographic breakdowns of their listeners.

For musicians, Pandoras Artist Marketing Platform (AMP) is data gold. Akin this to Facebooks advertising targeting metrics, or Power Editor, for the publishing world: target topics you cover to the people who have already expressed interest in such topics. Find look-alikes, too, no problem.

With AMP, we hope to make the day in and day out easier for artists by eliminating the guesswork, said co-founder Tim Westergren. From finding out what songs are performing well to inform singles or set lists, to mapping where an artists fan base is to inform tour schedules, our ultimate goal is to help artists across the spectrum build and maintain their careers.

For Pandora, this announcement helps the company to position itself in artists favor, as was Spotifys claim in December 2013. Both Pandora and Spotify have long received criticism for their low royalty pay-outs with some high-profile artists claiming that the platforms will suck all the creative content out of the world. With a data platform available to artists though, Pandora can offer value back to musicians, and even later connect that data to its advertising platform to better target to Pandora users.

Apples OS X Yosemite Collects User Location Data and Scans Hardware

Apples newest OS X software release is angering many, with claims that the company is collecting hardware search and location data without consent. Worse, the company is sharing such data with Microsoft, the company powering the new search tools on OS X Yosemite via Bing.

Overall, the move seems to be an attempt for both Apple and Microsoft to collect more data in attempts to rival Googles global status at the worlds largest data broker.

For the most part, Apple and Microsoft will likely only use this data for advertising and marketing purposes, serving up for relevant, personalized ads to users. Not such a bad deal. However, there are concerns that the data collection could be shared with government entities, as similar data collection has been done in the past.

That said, while it is an automatic opt-in (which Umbel does not endorse), users can opt-out. Heres how, thanks to The Independent:

Go to System Preference> Spotlight > Search Results and disable the following: Spotlight Suggestions, Bookmarks & History and Bing Web Searches.

“The biggest issue here isnt in the data collection itself, but the lack of notification on Apples part on doing such.”

Apples statement on the issue was that the company is absolutely dedicated to protecting users privacy. And that the main Spotlight function that collects data (Spotlight Suggest it offers personalized results just like any web search engine) doesnt use a persistent identifier, so a user's search history can't be created by Apple or anyone else. The company also said that they blur geographical data so that no exact locations may be known.

The biggest issue here isnt in the data collection itself, but the lack of notification on Apples part on doing such. In protecting users data rights, companies need to be forthright about when and why they are collection data. On this topic, The Independent's own security and privacy researcher Runa Sandvik weighed in:

This removes some of the big, potential dangers with this type of ‘feature',” she said, adding that the company was still participating in a worrying trend in tech companies to “learn as much as possible about users, where they are and how they use your services.”

Very worrying, indeed.

Ogilvy Looks to Data for Creative Innovation

Ogilvy & Mather is no agency that sits on the sidelines. Revealed in a New York Times article on Oct. 20, the agency has created OgilvyAmp, a department of the company tasked solely with handling data strategy and planning, analytics services and data management. They are working for clients that include IBM, British Airways, E*Trade, Kimberly-Clark, Merck, Nestl and T-Mobile.

Ogilvys move toward helping clients understand and value their data comes from one overarching issue with Big Data itself: people are afraid to dive into it. Data literacy is extremely low, and the demand for those with Big Data talent and experience in high, while availability for those with it is quite low. In addition, media attention around the subject often either falls into one of two categories: that of fear-mongering or that of undue praise.

OgilvyAmp will help data serve as an integrator as the digital revolution dissolves the traditional boundaries between the disciplines like advertising, direct marketing, public relations.

So, where the CMO and CIO roles are overlapping, with current employees in both roles not fully understanding the new tasks assigned to them, Ogilvy is stepping up.

This is a huge opportunity for clients and for us, said Todd Cullen, who joined Ogilvy & Mather last year in the new post of global chief data officer, to remove data as a distraction and position it as a tool in the creative palette.

Ello Gets Funding, Makes Legally Binding Promise to Never Collect or Sell User Data

Ello, the anti-Facebook social platform that went viral about a month ago, gaining one million users with another 3 million registered and waiting for an invite, got a $5.5 million boost on Oct. 23 from a Series A round of venture capital funding.

The only caveat? The new social platform, which gained such notoriety based on a promise of never collecting, using or selling user data (a concept deemed insane by the likes of Google, Facebook, Twitter, LinkedIn and the vast majority of the internet), is now legally required to maintain that promise.

To do this, Ello founders registered the company as a public benefit corporation (PBC) in the state of Delaware, of which there are only about 1,140 in the entire country. In the charter to register, the founders wrote:

Ellos explosive growth over the last few months proves that there is a hunger to connect with friends and see beautiful things without being manipulated by ad salesmen, boosted posts, and computer algorithms that dont always have our best interests at heart. On an ad-driven social network, the advertiser is the customer and youre the product thats bought and sold.

A PBC is obligated to consider the mission based, Ello founder Paul Budnitz told Betabeat. We really cannot be forced by our investors to break the basic principles.

While this is most certainly a true statement, this move is leaving many others to ponder on the monetization of such a network. How, after all, can a digital company make money without selling ads, selling data or forcing users to pay a premium?

Thats the question yet to be answered. As of now, though, Ello is obviously a company dedicated to ethics, and that is a win in our book, especially when it comes to data rights.

Adobe and Nielsen Team Up And It Was About Time

Nielsen and Adobe are teaming up to finally, and we mean finally, provide viewership numbers and demographics on online TV audiences.

Nielsen is historically best known for providing TV show audience ratings, which help advertisers place ads during particular segments. However, this process has been limited to only broadcast television and has wholly ignored the growing online TV watching audience.

The partnership is likely a play at competing with the data housed by the likes of Hulu, which uses audience data to help advertisers on their site reach a relevant audience.

ESPN, Turner, Sony Pictures (including online video service Crackle), Univision, and others have already signed up to access the data.

The post Top Tech News You Need to Know: Pandora, Apple, Nielsen and More appeared first on Datafloq.

In the meantime, heres what you need to know about last weeks biggest stories.

Big Data Knows Who You Are, Where You Shop and More And Esri Can Prove It

Theres a new Big Data tool out on the market, and it can break down your idiosyncrasies based solely on your zip code. Thats right: type in your zip to Esris database, and the site will reveal the top three (of a total 67) consumer groups that particular area falls into.

Sound like 67 total consumer groups for all of the United States is making things a bit too, simple? Not so: the explanations for these groups have thus far proven to hit, um, a little too close to home for some. For example, Catie Keck, writer at Bustle, typed in her own Williamsburg, Brooklyn zip code, 11211, and received this response:

Dressed head to toe in the most current fashions, we fill our weeknights and weekends with discovering local art and culture, dining out, or exploring new hobbies. We must be connected at all times; texting and social media are essential for communication and keeping up with our social lives. E-readers and tablets are preferred for everything except womens fashion and epicurean magazines which must be in print.

Her comment on which was as follows:

Whoa, slow your roll there Esri before you hit the nail on the head too hard.

Others have been equally impressed with Esri, and the site gained news coverage from nearly all audience sector publications this week.

The level of detail is striking and from what I could tell based on cross-referencing some of my own last several zip codes of residence pretty accurate, too, wrote Adrienne Lafrance for The Atlantics CityLabs. Anyone can plug a zip code into Esri's database, which makes for an addictive game of guess my identity. The database is also a way to see how neighborhoods gradually change from one zip code to the next.

The real genius of the database, though, is in how marketers can use this information to better understand their audience and craft their messaging.

After all, weve always known that getting the attention of a resident of Manhattans Upper West Side and one that lives on Austins East Side is a whole different ball game. This database now just proves it.

Millennials are to Big Data as Peanut Butter is to Jelly

Not coming as too big of a surprise, but millennials as it turns out are totally cool with Big Data, and are much more data literate than they get credit for.

A full two-thirds of them report, in a new study, that they understand the types of data being collected about them, and how that data is used. And nearly 80% of young adults aged 18-30 feel in control of their personal online data, according to the findings of a Telefnica-commissioned survey of 6,700 Millennials across 18 countries.

The one place these data savvy adults do express concern though? Data privacy and security. A full 90% reported having taken active steps toward protecting themselves online.

We all agree that data is the lifeblood of digital technologies. It is therefore vital that the reform of the legal framework for data protection results in a trusted digital environment, said Dr Richard Benjamins, Telefnica's Group Director of Big Data. Policy makers should take a risk-based approach which considers not only how data is collected but also how it is used. They should aim to protect people first, rather than data, and must prevent the use of data in ways that might negatively impact individual peoples lives.

In all, millennials are digital and data optimists, but they are full and ready for to fight back if their data rights are compromised.

Youre Never Anonymous Online And Whisper is Proof

On Thursday, The Guardian released a story highlighting that Whisper, the social media app that promises users anonymity and claims to be the safest place on the internet, is tracking the location of its users, many of whom specifically opted out of location tracking.

Yikes.

The Guardian alerted Whisper to the upcoming story, an on Monday, three days before the story went live and four days after Whisper received The Guardians alert, the app updated its terms of service to include permitting the company to establish the broad location of people who have disabled the apps geolocation feature.

Just how specific does Whispers geotracking get? Well, it knows where you were within 500 meters of when you posted to the app.

The technology, for example, enables the company to monitor all the geolocated messages sent from the Pentagon and National Security Agency. It also allows Whisper to track an individual users movements over time, reported The Guardian. When users have turned off their geolocation services, the company also, on a targeted, case-by-case basis, extracts their rough location from IP data emitted by their smartphone.

The news trended on Twitter nationwide on Thursday after the story was released, with many users wholly leaving Whisper to create their own, actually anonymous apps. Buzzfeed announced that it is halting their partnership with the app until the privacy terms are cleared up. The Guardian itself has also ended its relationship with Whisper.

This news itself comes after two Guardian reporters were invited to the Whisper offices and shown the technology, with no one at the company expressing any concern for the privacy violations. The reporters were at no time told they could not report on the information being shared with them.

Whisper isnt actually about concealing identity. Its about a complete absence of identity, the companys co-founder and CEO, Michael Heyward, recently told Entrepreneur magazine. The concept around Whisper is removing the concept of identity altogether, so youre not as guarded.

Now, Whisper seems to be about the complete absence of trust.

The F.B.I. is Seriously Not Cool with Apple and Googles New Encryption Policies

James B. Comey, the director of the F.B,I., announced on Thursday that Apple and Googles encryption policies in this post-Snowden era have gone too far. In response, he hinted that the administration may enact their own policies forcing companies to create a way for the government to unlock photos, emails and contacts stored on phones.

Comey, however, failed to comment on critics questions concerning the ability for the NSA, terrorists, hackers, spies and the like to also, then, be able to access the same type of information. After all, if there were a way for Apple and Google to allow the government to get information off of a phone, then that method could then be hacked and used for non-law enforcement purposes.

F.B.I. agents see the encryption as a beachhead they cannot afford to lose, wrote David E. Sanger for the New York Times. With the latest software, the new phones will be the first widely used consumer products to encrypt data by default. If that is allowed to stand, investigators fear other technology companies will follow suit. If all desktop computers and laptops were encrypted, it would stymie all kinds of criminal investigations, they say.

Ten months ago, President Obamas advisory committee recommended the increased use of encryption and that the government encourage companies to do the same. Comey did not reference this that report and White House officials have said that they are still working on an official policy for Obama to approve.

Apple and Google have declined to respond to Comeys comments, however, Colin Stretch, the general counsel for Facebook, called encryption a key business objective for technology companies. Id be fundamentally surprised if anybody takes the foot off the pedal of building encryption into their products, he said.

The post Top Tech News You Need to Know: Whisper, The F.B.I. and More appeared first on Datafloq.

43% of Companies Experienced Data Breach in Past Year, Including Jimmy Johns, Home Depot, Target

Last wednesday, the Ponemon Institute, which does independent research on privacy, data protection and information security policy, released a study revealing that 43% of companies experienced a data breach in the past year up 10% from the previous year. And worse, the breaches are getting bigger.

American consumers are beginning to feel the effects of these breaches, especially when they hit popular retailers including Target, Home Depot and, most recently, Jimmy Johns. More than half of U.S. consumers, 68%, say that the NSA and recent retail breaches raises serious concerns for them over data security, though only 37% have chosen to use cash over credit and 41% have checked their credit scores to monitor for breach repercussions.

Internationally, breaches are affecting populations on a scale yet to reach the U.S. In South Korea, for example, 70% of South Koreans ages 15 to 65 a total of 27 million had their personal data stolen and credit cards compromised in January 2014.

As far as causes of data breaches, 80% of them stem from employee negligence, and that negligence is trickling down to a consumer level as well. For evidence, the Home Depot breach by far exceeded the severity of the Target breach, with 16 million more credit card numbers put in jeopardy, and yet consumers are less concerned about the implications.

Both stories have been covered extensively by the media, so what is the issue here when it comes to consumer care?

“Targets response [to the data breach] was epically bad, says Paula Rosenblum, the managing partner at RSR Research. Or, as attorney Stephen Lesavich, author of “The Plastic Effect: How Urban Legends Influence the Use and Misuse of Credit Cards,” puts it: “The senior management of Target did not handle the breach very well and that generated a lot of bad press.”

That said, 27% of companies still do not currently have a protocol in place for handling a potential data breach, though that number is down from 39% last year.

In all, data breaches are a part of our new, collective reality and consumers are looking toward brands to protect their data from hacks or properly alert them to breaches to help customers avoid intense breach repercussions.

Companies, it is time to stand up and take control of your data rights. Contact us to find out how.

Google Builds New Dutch Data Center for $770M

Google, one of the largest data brokers in the industry, is building a new data center in Eemshaven, Groningen. The center will cost about $772m over the next four years. Why The Netherlands for Googles newest, most hyper-efficient data center? Well, for one, climate.

“[Eemshaven] pitches itself as an ideal location for such things it has access to many kinds of renewable energy…its a landing station for international internet cables, and it has a mild climate thats conducive to cheaper cooling,” says David Meyer, senior writer for Gigaom.

This new center will be Googles fourth hyper-efficient data center in Europe. Googles WIlliam Echikson explained on Googles Europe Blog:

“The new Dutch data centre will benefit from the latest designs in cooling and electrical technology. It will be free-cooled – taking advantage of natural assets like cool air and grey water. … Our data centers use 50% less energy than a typical datacenter. … This will be Google's fourth hyper efficient facility in Europe. … Its much more efficient to build a few large facilities than many small ones.”

The facility is also expected to help bridge unemployment in Europe as well as compete with Microsoft, which reportedly is building a 17.5 megawatt data center in Noord-Holland, just south of [the area].

Save Your Data iOS 8s Most User-Friendly Feature

Apples new iOS 8 platform was pushed out this week to iPhone users and met with backlash as the download erased thousands of photos, music files and more from users phones. In general, the iOS 8 platform has been viewed by the industry as buggy and Apple pulled the update to fix the known and widespread bugs affecting many iPhone users, releasing a new version as of Thursday.

Despite the hubbub surrounding the iOS 8 platform, there is one feature for which many loyal Apple users have long been waiting: data use visibility.

Long-available on Android phones, the Apple iOS 8 platform now allows users to see which apps are sucking their data, allowing users to make better decisions as far as which apps to open, to download and so forth. You can also check to see which apps are sucking up your battery life as well.

In all, go ahead and wait to see if the bugs are fixed before downloading the new OS. Then, make yourself some data-driven decisions on how you use your mobile data and battery life. Data and battery life-sucking apps, beware.

Big Data and Recruitment: An End to Hiring Discrimination?

Big data may soon help to end hiring discrimination in the workplace. And the first industry to feel the effects may very well be the industry that needs it most: tech.

The workplace diversity at two of the largest tech companies in the country are historically and quite infamously mostly white and mostly male (looking at you Google and Facebook). Many organizations and startups have actively attempted to sway the status quo in favor of a more diverse tech atmosphere, for instance Tristan Walkers Code 2040, which pulls in minority kids, teaches them how to code and secures them internships at prestigious startups in Silicon Valley and New York City.

Yet, programs like Code 2040 and few and far between, and besides, even Google admits that there is an unconscious bias when it comes hiring.

Many in the big data industry hope that this unconscious bias can finally be beat once and for all with data-driven HR practices.

“Big data in the workplace poses some new risks, but it may yet turn out to be good news for traditionally disadvantaged job applicants,” said David Robinson, a principal at Robinson + Yu, a consulting group that works to connect social justice and technology.

Of course, many still have their reservations, especially considering that algorithms run off of historical data. Historically, history is the problem with the current workplace environment in tech.

“If you're a company that doesn't have a history of hiring women or minorities, your model will tell you that these people are not especially well qualified. Even if you're simply trying to minimize turnover, what you might do is systematically exclude certain groups that are poor or of a certain ethnicity, says Solon Barocas, who is conducting postdoctoral research on the ethics of machine learning at Princeton's Center for Information Technology Policy and just coauthored a study on big data's “disparate impact.”

On the other side of the coin, the FTC is cracking down on potential discrimination due to big data.

The FTC is “committed to rigorous enforcement” of current law related to data privacy and discriminatory practices, but companies, U.S. policymakers and other groups need to have a deeper discussion about fair big data practices, FTC Chairwoman Edith Ramirez said during an agency workshop on big data and discrimination.

“I don't think the [longtime] structures need to be reinvented or shoved aside because data sets are larger,” says Ramirez. “It's important to keep the regulations that we have … to ensure that fair information practices are still applicable and relevant.”

Investment in Big Data Continuing to Grow

Companies continue to invest in big data and the 2014 numbers show an ever-increased look toward data-driven business practices. Gartner Inc. found that 73% of the 302 Gartner Research Circle members worldwide have invested or plan to invest in big data in the next 24 months, up from 64% in 2013.

That said, the market is showing a slowdown of new platforms and startups into the space.

“The hype surrounding big data appears to be tempering down,” Ashish Nadkarni, research director, Storage Systems and Big Data research at IDC, said in a statement. “This is a sign that the technologies are maturing and making their way into the fabric of how organizations operate and firms conduct their business.

Image: Google

The post Top Big Data Tech News You Need to Know: Retail Data Breaches, iOS 8, Data-Driven HR and More appeared first on Datafloq.

In case you missed it, here are the top Big Data tech news stories from last week, just to get you caught up after the weekend.

92% of Execs Are Happy with Recent Big Data Outcomes

Still think Big Data isn't actually churning any ROI? Well, then you're in the minority. According to consulting firm Accenture's research, which surveyed more than 1,000 executives from large companies, 92% of execs said they are “satisfied with the results” from their Big Data projects, with 89% rating Big Data as “very important” or “extremely important” to their businesses' digital transformation.

OK, don't think “satisfied with” is good enough? Good thing, then, that 82% of C-level executives and company technology leaders agreed Big Data provides a “significant source of value” for their companies.

So, what are these data-driven execs using their Big Data to do? 84% said their companies use Big Data “moderately” or “extensively” to identify new sources of revenue, 90% use it to retain and acquire customers and 89% use Big Data to develop new products and services.

“Businesses are at a transition point where instead of just talking about the potential results that can be achieved from Big Data, they are realising actual benefits including increasing revenues, a growing base of loyal customers, and more efficient operations,” said Narendra Mulani, senior managing director of Accenture Analytics. “They're recognizing that Big Data is one of the cornerstones of digital transformation.”

Apple Summit Introduces New Tech And No, It Isn't a Device

Apple's two-hour presentation on Tuesday introduced a lot of new technology, most of which the public very much expected. There's a new iPhone just in time for the holiday season. Check. There's a new smart watch to bolster Apple's presence in the wearables market. Check. And there's a new tap-to-pay tech that has the potential to destroy the likes of PayPal.

Wait, what?

That's right: potentially the biggest news to come out of the Apple summit, so to speak, was news of Apple Pay, and the timing couldn't have been any more serendipitous. Following on the heels of the Home Depot credit card breach, which mimicked the same tactics used in the Target credit card breach of 2013 which led to Target's CEO stepping down Apple Pay is positioned as a safe, secure payment method that anonymizes your credit card number to decrease the chances that your credit card information is stolen in retail breaches.

Of course, given iCloud security issues raised with the nude celebrity photo hack, many aren't so sure that a tap-to-pay system is any more secure than a traditional credit card. However, experts are saying that to-date, Apple Pay does indeed seem to be more secure than our plastic cards.

“Your old fashioned credit card can be cloned by a waiter. Merchants routinely have credit card numbers stolen,” said Christopher Soghoian, a technologist at the American Civil Liberties Union who last week criticized Apple's iCloud security. “There's not going to be anyone who says Apple Pay is less secure than the old-fashioned swipe cards.”

The security of Apple Pay relies on a 16-digit encrypted number. Each card will get its own token specific to the device it is entered on. According to The Wall Street Journal, this is how it works:

“When a consumer wants to buy cheese at a Whole Foods, the device would use a short-range radio system, called near-field communication, or NFC, to send the 16-digit token to the card reader. This token would be paired with a one-time string of random numbers created by encryption keys stored on the iPhone.”

“It's impossible to do transactions without data, and data is obviously a potential risk,” James Anderson, group head of mobile product development at credit card processor MasterCard Inc., which has worked with Apple on the payment technology, told The Wall Street Journal. “We are hypersensitive to that topic .. Let's say somebody is listening and is able to pick up the data. Essentially you get a useless 16-digit number. I could email it to my friend and they could email it back to me, but there's nothing they could do with it.”

Looks like Apple is getting serious about solving the current state of data insecurity.

Europe's Starring Role in the Big Data Ethics Battle

If there is anyone out there fighting tooth and nail for data rights and data ethics, it is certainly the EU. In what is being dubbed the fight between the right to information and an individuals right to privacy, the European Court of Justice and Google are going head to head.

Since May's “right to be forgotten” decision, Google has moved to accommodate European regulators, quickly starting to implement the ruling, which the company said Tuesday has led to 120,000 requests to remove content from search results. And now, Google is embarking on a seven-city tour of the EU, looking to spark debate over the ruling earlier this year.

“If there is anyone out there fighting for data rights, it is certainly the EU.”

“We need to balance the right of information against the individual's right to privacy,” said Google Executive Chairman Eric Schmidt in opening remarks Tuesday. “So we convened a panel of genuine experts whose criteria and qualifications are amazing to talk to us about this.”

Google execs will be on the road, then, for the next two months, looking to gain insight into the issue at hand.

“We're committed to complying with the Court's ruling, but some of the requests we're receiving raise difficult ethical and legal issues,” a Google spokesman said. “The Advisory Council will give us guidance on how to think about the more complex removal requests, which will complement the guidance we receive from Europe's data-protection authorities.”

Big Data Gets Up Close and Personal Yes, Closer Than You Think

Big Data is quickly changing the landscapes of our daily lives, both on and offline. This includes the way we eat, work, think, play and, yes, even hook up. And now, thanks to Christian Rudder and OKCupid, we have the data to back it up.

And because numbers don't lie, not everything Big Data reveals about us is very pretty.

In his new book “Dataclysm, Who We Are When We Think No One Is Watching,” Christian Rudder, Founder of OKCupid, reveals nine revelations about sex and dating all backed by Big Data.

Here's what we know (again, we emphasize the not-pretty nature of these):

1. Straight men think women have an expiration date
2. Straight women are far less likely to express sexual desire than are other demographics
3. “Most men lead lives of quiet desperation and go to the grave with the song still in them”
4. Searches for “Is my husband gay?” occur in states where gay marriage is least accepted
5. According to Rudder's research, Asian men are the least desirable racial group to women
6. And black women are the least desirable racial group to men
7. Users who send copy-and-paste messages get responses more efficiently
8. Your Facebook Likes reveal can reveal your gender, race, sexuality and political views
9. Vermont doesn't shower a whole lot, relatively speaking

Check out this TIME article exploring the data behind these, or just buy Rudder's book to get the real insight.

Check Out 4 World Trade Center's Smart Elevators

It just wouldn't be right to let the week of Sept. 11 pass without a mention as a newsworthy topic. It is. Of course, let's focus instead on the positive that is coming out of Ground Zero.

Beyond 4 World Trade Center being one of New York City's greenest buildings, a feat in and of itself thanks to the Hearst Tower, it is the only building in the city with smart elevators.

The elevators use an algorithm that will learn if, say, a tenant typically visits two or three floors in succession at a certain point of the day and automatically route them that way with the swipe of an ID card.

“I could program that every time I leave my floor, I go to the 52nd floor, that's the cafeteria. I can present my badge and the elevator will automatically take me to the 52nd. From there, I want to go to the gym, it'll automatically do that as well,” Jerry Piserchia, Shindlers senior project manager, told Fox News.

As expected, security and safety are also very important to 4 World Trade Center.

“If desired by the tenant, we can feed that information back to them so they can track who is on what floor,” Piserchia said. “So they issue the credentials for who can get in the building, we take care of tracking where they are throughout. A lot of these features have been built in for a number of different reasons including security, preferences, and even in some cases like when SARS was going around Asia and people didnt want to be touching elevator call buttons.”

The post Top Big Data Tech News You Need to Know: Apple Pay, The Right to be Forgotten and More appeared first on Datafloq.

It is no surprise that also Apple deals with Big Data. As is with many of the products that Apple develops, there is however not that much known about how Apple deals with Big Data. But with over 50 billion of downloaded apps from the app store, Apple swims in massive amounts of data that can, and will be, analysed for additional insights. Like many of the tech companies in Silicon Valley they have been using Big Data for quite some time.

Although they are a bit more secretive than other companies like Google, who are more willing to share some of the Big Data innovations that they have created, Apple does apply some of the Big Data technologies like Hadoop or Large Scale Data Warehousing. One of the uses of Big Data is to inform the design of their application, as mentioned by Jeff Kelly, Principal Research Contributor at Wikibon. Using all the data that they have collected about how their applications are used on the iPod, iPhone, iPad or Macbooks and Macs they can test new features relatively easy and sometimes do A/B testing to improve the experience. They use data to understand how people are using their applications. When it is a game application they can use the data to understand where there is a bottleneck or where a lot of people get stuck. They will use that data to improve the gaming experience. In addition they use the feedback and reviews users provide in the Appstore to improve their products.

In order to store all that data, Apple uses Teradata equipment and reportedly they are the fastest customer of Teradata to reach Petabyte scale in a Teradata warehouse. They now operate a multiple-petabyte Teradata system, which is mostly driven by the launch of iCloud in 2011. Robin Harris believes that these petabytes are mostly used to store personal information. Of course, this was 2011 and two years later it is likely that this data warehouse has grown tremendously.

Although they are gearing up for the Big Data era fast, it does not mean that they have it all set out correctly. In an article on Forbes in 2012, it was reported that one of the likely causes of Apples failure in entering the Mobile Maps Application market was its struggle to catch up with Big Data. While Google already opened up its mapping functionality in 2005, Apple had to catch up in the game in much shorter timeframe, which revealed to be too short to develop a well-working product.

It is obvious that Apple deals with massive amounts of data. Each and every Apple product communicates back to the Apple data warehouse on a continuous basis. Think only the amount of data that is generated via Siri and stored for two years. But also the data collected via iTunes & iTunes Match, iCloud and usage data of all software and hardware. Although being a lot of data, it is nothing spectacular or unexpected. Something we are not used to of Apple. Could the rumored Apple Watch be the game changer in the Big Data field?

The rumoured Apple Watch could possibly be a data generator beyond any wearable device currently on the market. Storing data about how much you walk, how many calories you burn on a daily basis, how well you have slept and in the mean time constantly communicating with all your other Apple devices. Bernard Marr argues in an article that when many consumers on a global scale will wear such a smart watch it will generate massive amounts of data regarding a nations physical output, eating habits or risk indicators. If we look beyond the standard options, and Apple introduces environmental sensors such as accelerometers, thermometers, barometers, altimeters into the device we are talking serious data. Combing that data with personal data, location data as well as time and data (after all it is still a watch) would seriously mean Big Data. Of course, this is all speculation and we will have to wait if and when Apple introduces a smartwatch.

Clearly, Apple is working with Big Data for quite some time now. How much and what they do exactly is difficult to find out. Whether or not Apple will re-invent Big Data with the Apple Watch remains speculative as until the device is revealed everything remains unclear. One thing is for sure; Apple generates and collects vast amounts of data, uses several Big Data technologies and uses this in the development of products and services to improve the customer experience.

Images: Insideview.com & Slashgear.com

The post Will Apple Re-Invent Big Data? appeared first on Datafloq.