The Perilous World of Machine Learning for Fun and Profit 05-01

The Perilous World of Machine Learning for Fun and Profit: Pipeline Jungles and Hidden Feedback Loops

                                                                        George prototyping a machine learning model.

I haven't written a blog post in ages. And while I don't want to give anything away, the main reason I haven't been writing is that I've been too busy doing my day job at MailChimp. The data science team has been working closely with others at the company to do some fun things in the coming year.

That said, I got inspired to write a quick post by this excellent short paper out of Google,  "Machine Learning: The High Interest Credit Card of Technical Debt."

Anyone who plans on building production mathematical modeling systems for a living needs to keep a copy of that paper close.

And while I don't want to recap the whole paper here, I want to highlight some pieces of it that hit close to home.

Pipeline Jungles

Picture

George prototyping a machine learning model.

There was a time as a boy when my favorite book was George's Marvelous Medicine by Roald Dahl. The book is full of all that mischief and malice that makes Dahl books so much fun. 

In the book, George wanders around his house finding chemicals to mix up into a brown soup to give to his grandmother in place of her normal medicine. And reading this bit of felony grand-matricide as a child always made me smile.

Prototyping a new machine learning model is like George's quest for toxic chemicals. It's a chance for the data scientist to root around their company looking for data sources and engineering features that help predict an outcome.

A little bit of these log files. A dash of Google Analytics data. Some of Marge-from-Accounting's spreadsheet.

POOF! We have a marvelous model.

How fun it is to regale others with tales of how you found that a combination of reddit upvotes, the lunar calendar, and the number of times your yoga instructor says FODMAPs is actually somewhat predictive!

But now it's the job of some poor sucker dev to take your prototype model, which pulls from innumerable sources (hell, you probably scraped Twitter too just for good measure), and turn it into a production system.

All of a sudden there's a "pipeline jungle," a jumbled up stream of data sources and glue code for feature engineering and combination, to create something programmatically and reliably in production that you only had to create once manually in your George's-Marvelous-Medicine-revelry.

It's easy in the research and design phase of a machine learning project to over-engineer the product. Too many data sources, too many exotic and brittle features, and as a corollary, too complex a model. One trap the paper points out is leaving in low powered features in your prototype model, because well, they help a little, and they're not hurting anyone right? 

What's the value of those features versus the cost of leaving them in? That's extra code to maintain, maybe an extra source to pull from. And as the Google paper notes, the world changes, data changes, and every model feature is a potential risk for breaking everything.

Remember, the tech press (and vendors) would have you build a deep learning model that's fed scraped data from the internet's butthole, but it's important to exercise a little self-control. As the authors of the technical debt paper put it, "Research solutions that provide a tiny accuracy benefit at the cost of massive increases in system complexity are rarely wise practice." Preach.

Who's going to own this model and care for it and love it and feed it and put a band-aid on its cuts when its decision thresholds start to drift? Since it's going to cost money in terms of manpower and tied up resources to maintain this model, what is the worth of this model to the business? If it's not that important of a model (and by important, I'm usually talking top line revenue), then maybe a logistic regression with a few interactions will do you nicely.

Humans are Feedback Loop Machines

Picture

The graveyard at Haworth

In Haworth England, they used to bury bodies at the top of a hill above the town. When someone died, they got carted up to the overcrowded graveyard and then their choleraic juices would seep into the water supply and infect those down below, creating more bodies for the graveyard.

Haworth had a particularly nasty feedback loop.

Machine learning models suck up all sorts of nasty dead body water too.

At MailChimp, if I know that a user is going to be a spammer in the future, I can shut them down now using a machine learning model and a swift kick to the user's derriere.

But that future I'm changing will someday, maybe next week, maybe next year, be the machine learning system's present day.

And any attempt to train on present day data, data which has now been polluted by the business's model-driven actions (dead spammers buried at the top of the hill), is fraught with peril. It's a feedback loop. All of a sudden, maybe I don't have any spammers to train my ML model on, because I've shut them all down. And now my newly trained model thinks spamming is more unlikely than I know it to be.

Of course, such feedback loops can be mitigated in many ways. Holdout sets for example.

But we can only mitigate a feedback loop if we know about its existence, and we as humans are awesome at generating feedback loops and terrible at recognizing them. 

Think about time-travel in fiction. Once you have a time machine (and make no mistake, a well-suited ML model is pretty close to a forward-leaping time machine when it comes to things like sales and marketing), it's easy to jump through time and monkey with events, but it's hard to anticipate all the consequences of those changes and how they might alter your future training data.

And yet when the outputs of ML models are put in the hands of others to act on, you can bet that the future (and the future pool of training data with it) will be altered. That's the point! I don't predict spammers to do nothing about them! Predictions are meant to be acted upon.

And so, when the police predict that a community is full of criminals and then they start harassing that community, what do you think is going to happen? The future training data gets affected by the police's "special attention." Predictive modeling feeds back into systematic discrimination.

But we shouldn't expect cops to understand that they're burying their dead at the top of the hill.

This is one of my fears with the pedestrianization of data science techniques. As we put predictive models more and more in the hands of the layperson, have we considered that we might cut anyone out of the loop who even understands or cares about their misuse?

Get Integrated, Stay Alert

The technical debt paper makes this astute observation, "It’s worth noting that glue code and pipeline jungles are symptomatic of integration issues that may have a root cause in overly separated 'research' and 'engineering' roles."

This is absolutely true. When data scientists treat production implementation as a black box they shove their prototypes through and when engineers treat ML packages as black boxes they shove data pipelines through, problems abound.

Mathematical modelers need to stay close to engineers when building production data systems. Both need to keep each other in mind and keep the business in mind. The goal is not to use deep learning. The goal is not to program in Go. The goal is to create a system for the business that lives on. And in that context, accuracy, maintainability, sturdiness...they all hold equal weight.

So as a data scientist keep your stats buds close and your colleagues from other teams (engineers, MBAs, legal, ....) closer with the goal of getting work done together. It's the only way your models will survive past prototype.

View at the original source

How the Nepal Earthquake Happened? 04-29

How the Nepal Earthquake Happened?

A little before noon Saturday in Nepal, a chunk of rock about 9 miles below the earth’s surface shifted, unleashing a shock wave—described as being as powerful as the explosion of more than 20 thermonuclear weapons—that ripped through the Katmandu Valley.

In geological terms, the tremor occurred like clockwork, 81 years after the region’s last earthquake of such a magnitude, in 1934.

Records dating to 1255 indicate the region—known as the Indus-Yarlung suture zone—experiences a magnitude-8 earthquake approximately every 75 years, according to a report by Nepal’s National Society for Earthquake Technology.

‘Earthquakes dissipate energy, like lifting the lid off a pot of boiling water. But it builds back up after you put the lid back on.’

—Lung S. Chan, geophysicist at the University of Hong Kong

The reason is the regular movement of the fault line that runs along Nepal’s southern border, where the Indian subcontinent collided with the Eurasia plate 40 million to 50 million years ago.

“The collision between India and Eurasia is a showcase for geology,” said Lung S. Chan, a geophysicist at the University of Hong Kong. The so-called India plate is pushing its way north toward Asia at a rate of about 5 centimeters, or 2 inches, a year, he said. “Geologically speaking, that’s very fast.”

As the plates push against each other, friction generates stress and energy that builds until the crust ruptures, said Dr. Chan, who compared the quake to a thermonuclear weapons explosion. In the case of Saturday’s quake, the plate jumped forward about 2 meters, or 6.5 feet, said Hongfeng Yang, an earthquake expert at the Chinese University of Hong Kong.Saturday’s quake was also relatively shallow, according to the U.S. Geological Survey. Such quakes tend to cause more damage and more aftershocks than those that occur deeper below the earth’s surface.

After an earthquake, the plates resume moving and the clock resets. “Earthquakes dissipate energy, like lifting the lid off a pot of boiling water,” said Dr. Chan. “But it builds back up after you put the lid back on.”

Nepal is prone to destructive earthquakes, not only because of the massive forces involved in the tectonic collision, but also because of the type of fault line the country sits on. Normal faults create space when the ground cracks and separates. Nepal lies on a so-called thrust fault, where one tectonic plate forces itself on top of another.

The most visible result of this is the Himalayan mountain range. The fault runs along the 1,400-mile range, and the constant collision of the India and Eurasia plates pushes up the height of the peaks by about a centimeter each year.

Despite the seeming regularity of severe earthquakes in Nepal, it isn’t possible to predict when one will happen. Historic records and modern measurements of tectonic plate movement show that if the pressure builds in the region in a way that is “generally consistent and homogenous,” the region should expect a severe earthquake every four to five decades, Dr. Yang said.

The complexity of the forces applying pressure at the fault means scientists are incapable of predicting more than an average number of earthquakes that a region will experience in a century, experts say.

Still, earthquakes in Nepal are more predictable than most, because of the regular movement of the plates. Scientists aren’t sure why this is.

The earth’s tectonics plates are constantly in motion. Some faults release built-up stress in the form of earthquakes. Others release that energy quietly. “Some areas, like Nepal, release energy as a large earthquake, once in a while,” said Dr. Chan. “These regions all have different natures for reasons geologists don’t really know.”

View at the original source

Wearable Device Helps Vision Impaired Avoid Collision 04-29

Wearable Device Helps Vision Impaired Avoid Collision

People who have lost some of their peripheral vision, such as those with retinitis pigmentosa, glaucoma, or brain injury that causes half visual field loss, often face mobility challenges and increased likelihood of falls and collisions. As therapeutic vision restoration treatments are still in their infancy, rehabilitation approaches using assistive technologies are often times viable alternatives for addressing mobility challenges related to vision loss.

Researchers from Massachusetts Eye and Ear, Schepens Eye Research Institute used an obstacle course to evaluate a wearable collision warning device they developed for patients with peripheral vision loss. They found the device may help patients with a wide range of vision loss avoid collisions with high-level obstacles. Their findings are featured on Investigative Ophthalmology and Visual Science (IOVS).

“We developed this pocket-sized collision warning device, which can predict impending collisions based on time to collision rather than proximity. It gives warnings only when the users approach to obstacles, not when users stand close to objects and not when moving objects just pass by.

 So, the auditory collision warnings given by the device are simple and intuitively understandable. We tested the device in a density obstacle course to evaluate its effect on collision avoidance in people with peripheral vision loss. To show its beneficial effect, we compared the patients’ mobility performance with the device and without it. 

Just demonstrating the device can give warning for obstacles in walking would not prove the device is useful. We have to compare with a baseline, which is walking without the device in this case.” said the senior author Gang Luo, Ph.D., Associate Scientist at Mass. Eye and Ear/Schepens, and Assistant Professor of Ophthalmology at Harvard Medical School.

Gang Luo, Ph.D., Associate Scientist at Mass. Eye and Ear/Schepens, and Assistant Professor of Ophthalmology at Harvard Medical School, adjusts the wearable device that his team created to help those who are visually impaired avoid collision while walking. Image credit: Peter Mallen, Mass. Eye and Ear.

Twenty five patients with tunnel vision or hemianopia completed the obstacle course study and the number of collisions and walking speed were measured.

Compared to walking without the device, collisions were reduced significantly by about 37% with the device and walking speed barely changed. No patient had more collisions when using the device than when not using it.

This video shows a pilot testing of a portable collision device developed for visually impaired. A normally sighted person was wearing a goggle with a small opening, which gave him only 10-deg field of view. A warning device in his pocket gave auditory warnings when imminent collision risk was detected.

“We are excited about the device’s potential value for helping visually impaired and completely blind people walk around safely. Our next job is to test its usefulness in patients’ daily lives in a clinical trial study.” Dr. Luo said.

This video demonstrates an end-user prototype of collision warning device we developed for visually impaired and blind people.

This study is entitled Evaluation of a portable collision warning device for patients with peripheral vision loss in an obstacle course. Other authors are Shrinivas Pundlik and Matteo Tomasi.

View at the original source

Bluetooth Disc Plays Your Digital Music Like a Vinyl Record 04-29

Bluetooth Disc Plays Your Digital Music Like a Vinyl Record

JESSE ENGLAND HAS an evocative way of describing record players: He considers them “altars” for music. “There’s no contemporary media environment that I know of in the past 50 years that requires that amount of reverence and that amount of care,” he says.

It’s hard to argue with him. Compared to Spotify and iTunes, playing vinyl is an elaborate process: You have to flip through your collection, select a record and carefully remove it from the sleeve. If you’re really obsessive, you’ll give it a wipe with record brush before placing it on the turntable and delicately lowering the stylus. Listening is similarly involved: You have to be careful not to bump the table, and you’ve got to get up every 15 to 20 minutes or so and flip the disc. 

“Universal Record,” England’s latest project, revives that ritual for the modern age. It’s a vibrating plastic disc that lets you play music from any digital source, via Bluetooth, on any record player. It’s a clever hack, but it’s even more interesting as a piece of media archaeology, focusing our attention not on the sound quality of vinyl but on the experience of using it.

England, who lives in Pittsburgh, Pennsylvania, and holds a MFA degree from Carnegie Mellon University, is fond of examining media through unlikely technological mash-ups. With “Sincerity Machine,” he modified a vintage typewriter to print text in Comic Sans. He used a laser engraving machine to cut letterforms in acrylic and glued them onto the typewriter’s strikes. Universal Record was even more straightforward: Inside the chunky plastic disc is a transducer—essentially “a speaker without a cone,” England says—which receives audio via Bluetooth. The vibrations from the transducer are picked up by the stylus. Thus, through vibration, the digital music is made analog.

The point isn’t simply to fetishize vinyl. “I am critical of unbridled and unchecked nostalgia as a marker of credibility,” England says. Rather, it’s about exposing folks to different ways of seeing and listening. The project encourages us to consider both what’s lost and gained as we shed old forms of media. I doubt anyone would think to describe iTunes as an “altar” to music (other than a sacrificial one, perhaps).

You don't have to burn e-books; you just need to get the right person to type a command, and all the copies disappear.

England thinks there’s additional value in reacquainting people with vinyl—a more permanent form of media whose embedded values are increasingly rare in our dematerialized, digital world. “I am concerned about the trend of all media and all information, by extension, only being available through a networked source,” England says. “I think the experience of listening to music today is exponentially richer and more fulfilling in terms of access and the sheer amount of content, but with that there’s an associated volatility.”

With services like Netflix and Spotify, entertainment is built on the ever-shifting tectonics of licensing agreements. For music listeners, it’s easy to imagine rare material falling through those cracks. But the problems are even more profound when you consider societies in which the free flow of information isn’t a given. Networked media is dangerously susceptible to censorship and suppression. You don’t have to burn e-books; you just need to get the right person to type a command, and all the copies disappear. “I’m concerned with people in any part of the world being comfortable with media being redefined from something that you have into something that you have access to,” England says.

Vinyl records, England says, have a built-in reliability. “You can play them back with the most basic of playback tools. You can just take a needle and a paper cone and you’d be able to recall the sound.” And how is the Universal Record’s sound quality? “All in all, it’s listenable,” England says. “But it really exists for the concept.”

View at the original source

Transforming an Analog Company into a Digital Company: The Case of BBVA 04-29

Transforming an Analog Company into a Digital Company: The Case of BBVA(Banco Bilbao Vizcaya Argentaria) 

In the past few years we have witnessed the far-reaching effects of the ongoing technological revolution on the ways we do business. Individual industries and whole sectors have been transformed; companies seemingly conjured from thin air swiftly rise to the top of their fields, joining the ranks of the world’s most valuable businesses. Conversely, long-established industry names fall into decay or disappear altogether.

This book provides a glimpse of the severe shocks that today’s companies are called on to withstand. Technology has changed, shifting the boundaries of production and distribution possibilities. Customers have changed, as have their requirements and the ways in which we reach them. Employees have changed, and their skills and motivation are now different. Change also takes place in organizational structures, decision-making models and forms of leadership, to meet the challenges of today and face those of tomorrow: technological progress and social development never stop, creating new uncertainties on the horizon of the business world.

These processes of transformation are all the more far-reaching, swift and radical in information-rich domains, such as the media, culture, and entertainment. Banking has changed, too. But despite being an information-rich activity—the “raw materials” of financial services are money and information—banking has changed a lot less than other industries. Money is readily digitized: when it takes the form of electronic book entries, it becomes information that can be processed and transferred in an instant.

Various reasons have been suggested to explain why banking has changed relatively little. First, the industry is subject to heavy regulation and government intervention. This discourages potential new entrants, so incumbent banks feel less pressure to change. Another factor often pointed to is average user age, which is higher than that seen in other industries—such as music. What’s more, most people take a conservative approach to their finances. And it may well be that the rapid growth and high earnings of the financial services industry in the years leading up to the downturn nurtured complacency and inefficiencies which in other sectors would have proved fatal.

But all this is changing. In fact, it already has changed. After the downturn, the financial services industry finds itself in an entirely new landscape. Laws and regulations are a lot tougher in the fields of consumer protection, good practice requirements, control, and capital ratios. This means thinner margins, higher costs, and lower earnings. In addition, users are now more demanding—they want improved transparency, cheaper prices and higher service quality.

Only a major effort of transformation will enable banks to return to profit figures capable of assuring medium- and long-term survival, and, by offering a wider, improved range of services at competitive prices, to restore their tarnished reputations in the eyes of customers and society at large.

This transformation is increasingly urgent for two powerful reasons. First, customers are changing swiftly; secondly, new competitors are stepping onto the stage.

A whole generation of customers have grown up with the internet—they make intensive use of social media and live in a “digital mode.” The “millennial” generation—also known as Generation Y—are now aged 25 to 40. They are approaching the peak of their professional development and making major financial decisions. By 2020, “millennials” will account for a third of the population of the United States and 75% of the workforce. 90% of them deal with their banks exclusively online, and half of them do so using their smartphones.

Over 70% of millennials say they would be happy to pay for banking products and services provided by non-banking companies—for example, telecommunications operators, technology and internet providers, online retailers. These percentages exceed 50% even among earlier generations—aged up to 55 years.1

What this means is that banks are losing their monopoly over people’s financial trust. And later generations—like “Generation Z,” born in or after the 1990s—will no doubt bring still greater developments which are yet to be discovered.

The United States is in most respects at the forefront of these changes, but the trend is global. It is not only in developed countries where we can see this shift. In developing countries, too, the more affluent customers are following the same pattern. What’s more, technology is making it possible to offer financial products and services to a poorer, more scattered population which conventional banks are unable to cater for at affordable prices. This potential market encompasses up to two billion new customers.

Change is opening up opportunities that foster the rise of a new league of competitors—mostly but not exclusively spilling over from the digital world. These new entrants can be far more efficient and agile than banks, because they are not burdened with inefficient, rigid and largely obsolete technologies or expensive brick-and-mortar distribution networks.

From Analog to Digital: Towards Knowledge-Driven Banking

Today banks must face a tough climate: tighter margins; overcapacity; tarnished reputations; and the pressure of new high-tech competitors who can move flexibly, unburdened by cost legacies.

But banks do enjoy a key competitive advantage: the huge mass of information they already have about their customers. The challenge is to turn that information into knowledge, and use the knowledge to give customers what they want.

It need hardly be said that the first thing customers want is better, quicker service on transparent terms and at an affordable price, in keeping with their own individual needs.

One of the implications is that customers should be able to interact fully with their bank, at any time and at any place, using their mobile devices. Today, there are 5 billion mobile phones in the world but only 1.2 billion bank customers. And mobile devices support an ever-increasing range of functionalities. Mobile data traffic now stands at more than 2.5 exabytes per month, and will almost treble every two years.2

This means on the one hand that the role of bank branches has radically changed; on the other, the potential scope of the banking market has widened immensely.

Facing new competitors who can move flexibly, unburdened by cost legacies, banks do enjoy a key competitive advantage: the huge mass of information they already have about their customers.

In the years to come the mobile phone will win a far greater share of interactions with banks. Technological progress—APIs, cloud computing—and increased investment in mobile banking development (now standing at about $2 billion a year in terms of venture capital alone, plus the heavy internal investment of banks themselves) will lead to a powerful rise in the operational features of mobile devices and in the range and complexity of financial transactions they will support.

Nevertheless, many people still want to deal with their bank by other means: branch offices, ATMs, computers, conventional telephones, and an increasing number of “smart” devices. So banks need to offer their customers a genuinely “omnichannel” experience. The same value proposal, the same service, must be available at any time by any channel, and you should be able to switch from one channel to another instantly and seamlessly.

And of course customers will increasingly want their bank to offer content carrying higher value-added—products and services that fit their needs more closely.

To meet these demands, banks must develop a new knowledge-based business model for the digital world.

According to Peter Weill, chair of the MIT Sloan Center for Information Systems Research (CISR), 3 the new digital model has three mainstays: first, content, the things being sold; secondly, customer experience—how the product or service is presented and used; and, thirdly, the technology platform, which shapes production and distribution.

I like to explain the construction of this new model by analogy to building a house. The technology platform is the foundation, while internal processes, organizational structures, and corporate culture are the various floors, including the installations (insulation, electricity, heating, plumbing, etc). Finally, the channels by which customers interact with the bank are the roof of the house. All these elements together make the house comfortable and safe. They let us offer the customer a good product and a satisfying experience.

For many banks, the technology platform is a limiting factor and a nearly insurmountable challenge. Most bank platforms were designed and built in the 1960s and 70s. Professor Weill calls them “spaghetti platforms,” because of the complexity of the connections resulting from several decades of add-ons, tweaks, and repairs.

This is why so many banks have tried to meet the digital challenge by building their “house” from the roof down, that is, starting with the channels. But that’s a stopgap solution. Without strong foundations, the increased volume and sophistication of online banking will overburden the obsolete platforms and the house will ultimately collapse.

This is because the improvement of content and customer experience to the standards customers want will call for systematic use of a vast volume of data.

The issue is not limited to handling an increasing volume of transactions and customer interactions. It crucially hinges on the huge amount of data collected in the course of customer contact, combined with the immense and rapidly increasing volume of information available on the internet, largely supplied by people’s social media activity and devices within the “Internet of Things.” We must capture, store and accurately process all that information to generate the knowledge to offer customers the best possible experience, even anticipating their needs and supporting them throughout their decision-making process. This is what I call “knowledge-driven banking,” which is far superior to what we now refer to as “customer-focused banking”—which in its time was a very meaningful improvement on conventional “product-based banking.”

Banks must take the lead in Big Data techniques if they are to make use of the competitive edge granted by their incumbent status. This can only be done with huge data-processing capabilities and a technological structure that fully and seamlessly integrates the knowledge thus generated with every customer channel and every point of contact.

Such capabilities are still beyond the grasp of conventional banking platforms. Cloud computing, however, has created the possibility of enhancing them flexibly and efficiently. Many of the new entrants to the banking field will use cloud computing, and it can be an immensely useful tool for incumbent banks as well. But security concerns and regulatory and compliance requirements call for a very careful decision as to which data, transactions, and capabilities ought to remain on the bank’s proprietary systems. The bank, what’s more, must coordinate and integrate all cloud-based services. This highly complex task will be powerfully aided by a flexible and modern technology platform.

Having said all this, the upgrading of technology, however necessary, is not the toughest challenge to which banks must rise. To succeed in the new digital world conventional banks must completely revamp their business model. We need to reinvent operations and processes, redefine organizational structures, undertake a revolution in approaches to work, and rethink the skills and talent we need our people to display. In short, we need a transformation in corporate culture, a complete reinvention of the business itself.

Transforming a conventional “analog” bank into a new “digital” provider of knowledge-driven services can only be a protracted and complex process. We must keep up an ongoing tension of testing and reassessing what we have, trial and error, an unending search. That is to say, we can never stop innovating. Our approach to work must accordingly be far more agile and flexible, less hierarchical, rich in communication across divisions, more open and more collaborative. The new culture we are called on to develop must be compatible with keeping up, to its full extent, the operational pace of our present business, our relationship with our customers and all our stakeholders. The process might be compared to changing the tires of a truck while still in motion.

Very few banks in the world have put themselves to the task with the necessary determination and depth. But our very survival is at stake. A new competitive landscape is taking shape in the financial services industry. A new ecosystem to which we must adapt.

Any number of these projects focus on transactions—payments, transfers, financial asset sales—such as PayPal, Dwolla, Square, M-Pesa, Billtrust, Kantox, Traxpay, etc. Adjoining this field we find companies that offer alternative currencies, such as Bitcoin, Bitstamp, Xapo, BitPay, etc.

Moving beyond the field of payments, initiatives are under way in other segments formerly monopolized by conventional banks: product and service selection advice (Bankrate, MoneySuperMarket, LendingTree, Credit Karma); personal finance management (Fintonic, Moven, MINT, etc.); investment and wealth management and advice (Betterment, Wealthfront, SigFig, Personal Capital, Nutmeg); crowdfunding capital and debt financing (Lending Club, Kickstarter, Crowdfunder, AngelList, etc.). Lending to individuals—so far thought of as the segment most resistant to disintermediation—is being addressed by the preapproved loans industry (Lending Club, Prosper, Kreditech, Lenddo and many others).

Some companies are even trying to extract value from banking transaction data itself by providing customers with APIs to access their data, or directly supplying the tools for any business to manage its financial transactions on its own or for a bank to develop its digital offering (Bancbox, Open Bank Project, Plaid, etc.).

The major online players (Google, Facebook, Amazon and Apple), leading telecoms companies and big retailers are taking a real interest in offering financial products to supplement their existing goods and services. There are several reasons for this. First, it enables them to offer their customers a fully rounded experience. Secondly, a financial relationship potentially entails multiple and recurring customer interactions through which a wealth of information can be extracted.

These players can supply a broader range of financial products and services and, eventually, create a fully fledged banking offer. At the very least they can create “packages” that combine their own products and services with financial products and services. These are packages that conventional banks will be hard-pressed to replicate.

Transforming a conventional “analog” bank into a new “digital” provider of knowledge-driven services can only be a protracted and complex process. We must keep up an ongoing tension of testing and reassessing what we have, trial and error, an unending search. We can never stay nnovating

We are witnessing the emergence of start-ups that focus on single segments of the value chain. These new entrants use the latest technology and lean, flexible structures to offer highly specific products. They can do so at cheap prices and offering a great customer experience by dint of speed, agility, and intensive use of Big Data technologies.

We are witnessing the disaggregation of the financial services industry, with a multitude of highly specialized competitors operating in different segments. What’s more, major players are likely to enter the market with wider product ranges. So the banking industry—clearly burdened by overcapacity and in need of a far-reaching process of consolidation—will see an influx of competitors who will put still more pressure on incumbent banks’ growth potential and bottom line.

Those banks that let the challenge of transformation go unmet, or fail in the attempt, are doomed to disappear. This won’t happen straight away. The regulatory framework is still a formidable barrier to some areas of banking, and many customers remain attached to established practice.

But these barriers will undergo an inexorable decline. Disintermediation in an ever-widening portion of the value chain will drive out the incumbent banks, leaving them with the heavily regulated areas only. Elsewhere they will be relegated to providing back-end tasks and mere infrastructure, at a distance from the end-customer.

Yet those banks that successfully achieve transformation will leverage their knowledge of their customers to remain as the main point of contact, offering a wider and better range of services, whether sourced internally or through platforms where various specialist providers and customers themselves can interact. Only the platform owner will be able to integrate all the knowledge generated about end-customers to enhance their experience and widen and improve the available product range (whether produced by the owner itself or other suppliers which the owner admits to the platform).

None of this is “economic fiction”—this ecosystem model is already a reality in the digital world. It is yet to reach the domain of banking in anything more than embryonic form,4 but its emergence is inevitable. At some point in the course of this process there will arise a competitive face-off between “digital” banks and the leading internet-based providers. The banks, using the financial information available to them supplemented by public sources, will seek to offer a wider and better range of financial and non-financial products. Their rivals will use their information about their users to offer financial services among others.

Against a background of swift technological progress and the rise of all kinds of new competitors, it will be hard to tell who is making the right decisions and implementing them tenaciously and imaginatively. Customer behavior will ultimately cast its light on the landscape. Those perceived as leaders in digital transformation will earn better prospects of growth and profit, which in turn will win them the technical and financial capabilities to make best use of the process of consolidation: they will attract the best talent, bolster their reputation facing customers and suppliers, and capture a larger, wider and truly global market share.

The Transformation of BBVA

At BBVA we soon became aware of the depth and reach of the change faced by the banking industry when many still thought our field would be perpetually shielded by regulations and user conservatism.

Seven years ago we undertook the task of rebuilding our technology platform from scratch. We entirely transformed our technology function so that we could at one and the same time keep existing systems in full operation and develop new systems in line with the latest technological advances. We doubled our systems investment from €1.2 billion in 2006 to €2.4 billion in 2013. A substantial change took place in the proportion of funds spent to keep systems operational (“run”) to funds invested in new development (“change”), moving from the industry standard of 80%/20% to a new standard of 60%/40%.

After seven years of work, at BBVA we have achieved a state-of-the-art technology platform.5 As a result, while we processed 90 million transactions a day in 2006, we were able to process 250 million transactions a day by 2013. We estimate we will reach 1.2 to 1.4 billion transactions by 2020. At the same time, the new platform enables us to meet increasing security requirements. From 2006 to 2013, the number of attempted attacks on BBVA multiplied by a factor of 60. However, technology-driven fraud in 2013 was less than half what it was in 2010.

In short, our technology platform is able to satisfy the requirements of data capture, storage and management, which are growing exponentially in step with our progress into the digital age. We are aware that this task can never be achieved completely. There will always be new and more complex demands. But we also believe we are ahead of our peers, and able to compete successfully with new digital entrants.

Technology, though essential, is only a tool in the hands of our people to help them build a better experience for customers.

What is needed here is a revolution in operations, processes, and organizational structures; a major shift in approaches to work and in the required skills and talent. This must signify a radical transformation of our corporate culture or, following our earlier analogy, the way in which we build the “floors” of our digital “house.”

From the outset we identified a number of essential cultural traits that we had to encourage: agility, flexibility, the primacy of collaborative work, an entrepreneurial spirit, and support for innovation. Given this, it fell to us to advocate open innovation models as a way to overcome the limitations to which organizations are typically subject, and place the development of value proposals in the hands of the best talent—whether it be found in employees, customers, outside partners or any other of the company’s stakeholders.

The cultural transformation is undoubtedly even harder to achieve than the technological one, because we lack any obvious model or benchmark. We have to work with the infinite complexities of people, social relations and pre-existing cultures.

After seven years of work, at BBVA we have achieved a state-of-the-art technology platform. As a result, while we processed 90 million transactions a day in 2006, we were able to process 250 million transactions a day by 2013

Over the past few years, at BBVA we have comprehensively re-engineered our processes in step with our technological overhaul. And we have promoted a change of culture. To spread the new culture and help it gradually permeate the entire organization, three approaches proved particularly useful to us.

First, leadership and top-down role models. At every public or internal presentation, the senior management of the Group stressed the need to embrace change, encourage innovation, and engage in collaborative work. We had to lose our fear of failure—which can be a rich source of learning and a driver of creativity. This approach was coupled with a major effort of internal and external communication on the bank’s developments in the digital domain. We set up models to be followed by raising awareness of our strategies and forward steps in this field, and of the people undertaking them.

Secondly, we leveraged our selection and training policies. We have invested more than €40 million a year in this area. The bank’s training division operates physical venues that serve as a powerful point of reference for all our employees and stakeholders, and enable us to share experiences and knowledge. Our training centers teach the key subjects involved in addressing change at BBVA (strategy, marketing, finance, technology, leadership) in partnership with external institutions that are at the forefront of their respective fields: e.g., London Business School, IBM, Center for Creative Leadership, Wharton, Harvard, IESE, IE, Boston College.

Our greatest effort, however, was to build one of the most innovative e-learning platforms in the world. This system enables us to provide over 3 million hours of online education and training, involving more than 175,000 course-takers (i.e., an average of 1.7 courses were taken annually by each Group employee).

New technologies (e-learning platform, use of mobile devices) and new learning approaches (newsletters, on-the-job learning, MOOCs) are becoming an increasingly important way of providing a range of flexible training options that everyone can access.

In the field of selection our goal is to be present wherever the talent and knowledge we need is likely to emerge. We have relationships in place with the leading, cutting-edge business schools, such as those referred to earlier and others. Our key “brand” as an employer is aligned with the goal of being “the best—and first—digital bank in the world.” We are leveraging intensive use of social media (LinkedIn, Facebook, Twitter) to achieve global positioning in keeping with our requirements and expectations.

Finally, we regard the new corporate buildings now under construction as powerful instruments to accelerate change. In the countries where we have a significant presence we are bringing together BBVA employees at new headquarters. Though originally driven by financial and efficiency -related criteria, this effort is now a lever in the service of achieving our transformation.

Over the past few years, at BBVA we have comprehensively re-engineered our processes in step with our technological overhaul. And we have promoted a change of culture

In this book the article produced by the BBVA New Headquarters Team6 provides a detailed account of how this project was implemented and what it was intended to achieve. Here I shall do no more than stress the point that our aim is to create a new work experience for the digital era. The design of this experience must be both global and focused on people, meeting their functional and emotional needs. Under the new approaches to work we aim to put in place the key vector is collaborative work as a way to bring forth collective intelligence and stimulate innovation. This requires simultaneous action in three distinct but interrelated realms: physical space, technology, and culture (behaviors). Given the interrelatedness of the three settings, the new headquarters are proving to be powerful drivers of behavioral change.

Over these years BBVA’s transformation drive has already garnered meaningful results. The Group’s active digital customers in December 2011 numbered 5 million; by mid-2014, that figure had climbed to 8.4 million. Active customers using mobile technologies grew from 0.3 million to 3.6 million (i.e., they multiplied twelvefold). We have reconfigured our branch network, too. On the one hand, we created small “convenience” branches focusing on customer self-service; on the other, we operate larger branch offices where financial specialists can provide customers with personalized advice and higher value-added. These developments and the introduction of a system supporting remote personalized advice have enabled BBVA to raise the average office time spent on sales efforts from 38% to 45%, while the proportion of sales staff to total Group employees has risen from 28% to 38%, coupled with significant growth in our cross-selling success rate.

We regard the new corporate buildings now under construction as powerful instruments to accelerate change

We have also set in motion a highly ambitious Big Data project. After a period of getting things ready and attracting the right talent, the initiative is enjoying real success as to customer segmentation, improved credit risk scoring, and fraud reduction, among other areas.

We have taken steps to encourage the emergence of an open innovation platform and community. Developers come together to present, critique and improve their ideas, and help one another develop new concepts and prototypes in a process of co-creation.

BBVA is already launching new products designed and produced specially for the digital world, such as BBVA Wallet and Wizzo. The new products are a good “test bench” and have proved a powerful way of building teams and helping them learn. They are also doing very well in the market.

While building our own capabilities and talent, we also keep an eye on outside talent. BBVA Ventures is a San Francisco-based venture capital firm with a global reach that invests in start-ups that develop innovative financial services.

BBVA Ventures helps us stay on top of what is happening in the realm of digital banking. It also enables us to form alliances involving promising teams and initiatives, and may open the door to acquiring talent, technologies or business models with a disruptive potential in the industry. Specifically, this was the case with our recent acquisition of Simple, a pioneering start-up that focuses squarely on the user experience in mobile banking. Our active presence in the domain of digital start-ups and our bid to support open platforms have led to another first of its kind deal: the recent agreement between BBVA Compass with Dwolla, a payment system start-up. Bank customers can now use Dwolla’s real-time payment network to make their money transfers.

These are all meaningful forward steps that carry the organization and its people onward to a new corporate culture. In fact, at BBVA we are confident of having gained a competitive edge over competing conventional banks both technologically and in terms of revamping processes, organizational structures, and corporate culture.

But the pace of change in the digital realm and the ongoing acceleration of the innovation cycle prompted us to speed up our transformation and turn around our organizational structure in radical ways to place the digital world at the center of our vision for the future. It was for this reason that in 2014 we created the Digital Banking Area.

BBVA Digital Banking: a Radical Organizational Change to Accelerate Transformation

The main purpose of this new business Area is to speed up the Group’s transformation into a digital bank. The Area is directly responsible for developing existing distribution channels, adapting internal processes and designing a new range of digital products and services capable of delivering the best possible customer experience.

The guiding idea is that the Area will, in addition to enhancing BBVA’s digital business and presence, work as a catalyst to transform the entire Group. In step with the digitization of the business, the Digital Banking Area will spread throughout every unit of the organization the relevant procedures, work methods, and culture. BBVA Digital Banking is the “BBVA version” of what John Kotter—in his paper collected in this book—has called the “dual model” for driving change forward.

The new Digital Banking Area’s mission has four key angles: a new, enhanced customer experience; knowledge-driven personalization using the best data analysis technologies; communication in clear, concise language; and access to products and services at any time and from any place.

This mission specification calls for a new set of behaviors. First, the customer must be the focal point of every decision. Secondly, we must be able to experiment and react quickly, launch initiatives and end them promptly if unsuccessful, and use the iteration method to improve. Thirdly, we must target what’s truly important for customers and the business—do fewer but more relevant things, and do them better. Fourthly, we must be accountable: we should set specific, quantifiable objectives, constantly measure our forward and backward steps, take any necessary corrective action promptly, and quickly take stock of any failures.

BBVA Digital Banking brings together all digital ventures and initiatives throughout the Group and, in light of the above principles, drives forward an ambitious project for change, which has two branches: transformation in project management and transformation in human resource management.

In project management we have opted for the Agile methodology. In a nutshell, this involves forming time-limited multidisciplinary teams specifically tailored to the requirements of the given project; team members make intensive use of online tools to work collaboratively. To allow them the widest flexibility in their approach teams are given full decision-making independence—although they are of course accountable for their decisions—and use iterative trial and error to test, improve, or reject the initial concept.

This new project management model entails a new human resource management model. The Digital Banking Area has been granted full authority to make its own hiring decisions. But we also want to encourage the entire Group to develop, search for, and recognize talent both within the organization and outside. We need our people to have the right know-how but also, and more importantly, we need them to have the cultural features that fit in with this new model—flexible, non-hierarchical, highly mobile, and subject to very quick project progress assessment cycles.

The Area is sub-divided into several functional units: Marketing, Customer Experience and Business Intelligence, Omni Channel, Technology, Strategy and Planning, and Talent and Culture. These units provide support for six business divisions. Four of those divisions are geographical, corresponding to the Group’s main regions: Spain and Portugal, United States, Mexico, and South America.The other two divisions are global: Forms of Payment and New Digital Businesses.

BBVA Digital Banking brings together all digital ventures and initiatives throughout the Group and drives forward an ambitious project for change, which has two branches: transformation in project management and in human resource management

This organizational structure reflects the Digital Banking Area’s goal of trans- forming BBVA’s existing activity and finding new, knowledge-driven lines of business in the digital realm.

In the business units, objectives—and, accordingly, decisions and priorities—are shaped by the features of each market, particularly the extent of its digitization and the specific opportunities available.

So in the more developed markets, such as Spain and the United States, the Digital Banking Area’s scope of action extends to digital transformation of the entire franchise and the development of a new business model. This means the Digital Banking Area is directly responsible for the entire market offering, the distribution model, and process design. As a result, the Area takes on joint responsibility with the local business unit for the income statement as a whole.

In Mexico and South America, the focus lies on developing the digital offering, whether at the request of the local unit or on the Area’s own initiative. In addition the Area manages all digital functions—product development, channels, marketing, processes, technology, etc. These efforts are aimed at the existing “digitized” population, but also seek to develop low-cost models capable of rolling out a profitable financial product range targeting low-income segments (financial inclusiveness). In these regions, the Digital Banking Area is co-responsible, in conjunction with the local business unit, for the “digital” income statement.

The Digital Banking Area is still at an early stage of development. It is attracting and integrating talent, building teams, specifying projects, and creating ties, agreements, and operational schemes with other business and support areas within the Group.

In the field of New Digital Businesses the aim is to develop new business models and value proposals beyond the scope of conventional banking. This unit—which is 100% digital in its organizational structure and culture—is entirely independent from the bank and operates as a global business line with its own income statement. It takes its projects forward internally or in partnership with others with a view to maximize the return on BBVA capabilities and assets and external talent. New Digital Businesses is accordingly in charge of the BBVA Group’s interactions with the digital ecosystem. One of the entities reporting to it is BBVA Ventures. It is also in charge of executing BBVA’s mergers and acquisitions in the digital sector.

The Digital Banking Area is still at an early stage of development. It is attracting and integrating talent, building teams, specifying projects, and creating ties, agreements, and operational schemes with other business and support areas within the Group.

But even at this incipient phase results are coming through. The Area has already launched almost fifty Agile-driven projects across all business areas involving close to 500 people. One key concern is to initiate and accelerate new projects; another highly significant consideration is to continue projects scheduled or started previously. Highly ambitious targets have been set for 2015 which involve a very significant portion of Group human and technological resources.

Leaving this aside, however, I believe that the Area’s most meaningful impact is that its principles and work approaches are permeating the organization as a whole—through its ties with other areas, its visibility in internal communications, and the backing it finds in the organization’s leadership.

This is why I think that the creation of BBVA Digital Banking was a bold and insightful decision in aid of speeding up the Group’s digital transformation. And I also believe we will shortly see the tangible results of the projects now being set in motion and of the Area’s role as a catalyst for change in the organization’s working approaches and culture.

Final Thoughts: Change Requires Leadership

It was some time ago that BBVA perceived the risks and opportunities inherent in technological change, and for several years we have worked towards reinventing ourselves and moving on from analog banking—however efficient and profitable it might have been by the standards of the twentieth century—into a knowledge-driven digital business of the twenty-first century.

This article focuses on two of the milestones in this process. First, the construction over the past seven years of an entirely new technology platform capable of supporting the data capture, storage and processing requirements of digital banking, which are far more demanding than those seen in conventional banking. Our platform now places us clearly ahead of our peers.

Secondly, in the domain of cultural transformation we have worked hard on several fronts and undertaken many significant projects. Our overarching goal has been to shape an organization that nurtures change and innovation—not as ends in themselves, but as a means to deliver the best possible customer experience. Very recently, the launch of the Digital Banking Area has marked a turning-point in the transformation of our processes, structures, approaches to work, capabilities and mindset, in alignment with the demands of the digital world.

We have come far, and are now in a position to lead the process of transformation of the banking industry and so become the first—and best—knowledge-based bank, fully in alignment with the digital ecosystem.

But we are aware that there is still a long road ahead. Our transformation, as we now envision it, is in progress and far from complete. Far more importantly, technological change continues apace, and society is changing with it. We are witnessing the dawn of Big Data technology. The Internet of Things is only just taking off, but is set to grow exponentially. In these realms, as in so many others—some of which we can as yet barely imagine—“more is different,” in the words of Kenneth Cukier in his article for this book, “Big Data and the Future of Business.”

So we are running a race which, for as long as the present stage of scientific and technological progress accelerates, has no discernible finish line. If we are not to lose our way, if we are not to become complacent or resign ourselves to being second-best, we must modify people’s attitude to change—we must not merely accept, but embrace and promote change. This calls for strong and cohesive leadership throughout the organization. Our leaders must advocate change, encourage change by example, recognize those who support change, and take steps to remove the practices and structures that stand in the way of change.

This is surely the key to BBVA’s successful transformation so far. And this is the kind of leadership we need in future if we are to achieve our goal that BBVA should become the foremost figure in transforming the best of analog banking into the best of knowledge-based banking for the twenty-first century.

View at the original source