InsideBIGDATA Guide to

Big Data for Finance

by Daniel D. Gutierrez

BROUGHT TO YOU BY
 Big Data for Finance

Introduction to Big Data

for Finance Contents
According to the 2014 IDG Enterprise Introduction to Big Data
Big Data research report, companies are for Finance. . . . . . . . . . . . . . . . . . . . 2
intensifying their efforts to derive value
through big data initiatives with nearly half Retail Banking. . . . . . . . . . . . . . . . . . 4
(49%) of respondents already implementing Credit Scoring. . . . . . . . . . . . . . . . . . 7
big data projects or in the process of doing
so in the future. Further, organizations are Back Trading & Testing. . . . . . . . . . . 8
seeing exponential growth in the amount
Adopting Big Data for Finance. . . . 9
of data managed with an expected increase
of 76% within the next 12-18 months. With Security Considerations. . . . . . . . . 12
growth there are opportunities as well as
challenges. Among those facing the big Regulatory Compliance
data challenge are finance executives, as Considerations. . . . . . . . . . . . . . . . 13
this extraordinary growth presents a unique
opportunity to leverage data assets like
never before.
As the 3 V’s of big data: volume, velocity VOLUME
and variety continue to grow, so too does
the opportunity for finance sector firms to Terabytes
capitalize on this data for strategic advantage. Records
Transactions
Finance professionals are accomplished
Tables, files
in collecting, analyzing and benchmarking
data, so they are in a unique position to
provide a new and critical service—making Batch
big data more manageable while condensing Near time Structured
vast amounts of information into actionable Real time Unstructured
business insights. Streams Semi-structured

VELOCITY VARIETY
Source – Exist.com 2013

The proverbial 3 V’s of big data: volume, velocity and variety and what
constitutes each classification for the finance sector.

2
www.inside-bigdata.com | 508-259-8570 | Kevin@insideBigData.com
 Big Data for Finance

Although many perceive a financial industry

mindset that’s wary of change — it is the cautious Big data solutions that support
adoption of big data technologies such as cloud evolving business and regulatory
solutions and Hadoop using the open source
requirements by maintaining a
distribution model that will be transformative in
many fundamental ways. Further, financial services network of growing data stores will
firms that use business analytics efficiently and become invaluable in their ability to
effectually will thrive in the current uncertain and be used for multiple purposes and to
fluctuating markets, while those that do not are answer any query months or years
likely to founder.
from now.
Financial services firms are discovering many
innovative and strategic directions by consolidating
data traditionally managed in departmental silos 2. New regulatory and compliance
in order to more holistically analyze risk exposure, requirements are placing greater emphasis
comply with regulatory mandates, and perform on governance and risk reporting, driving
enterprise-level analytics. Traditional technologies the need for deeper and more transparent
such as relational database management systems analyses across global organizations.
make it challenging, many times impossible, to 3. Financial institutions are ramping up their
process growing volumes of data and make them enterprise risk management frameworks,
accessible, actionable and elastic to changing which rely on master data management
requirements in terms of advanced analytics. Big strategies to help improve enterprise
data solutions that support evolving business and transparency, auditability and executive
regulatory requirements by maintaining a network oversight of risk.
of growing data stores will become invaluable in 4. Financial services companies are looking to
their ability to be used for multiple purposes and leverage large amounts of consumer data
to answer any query months or years from now. across multiple service delivery channels
(branch, web, mobile) to support new
predictive analysis models in discovering
consumer behavior patterns and increase
conversion rates.
5. In post-emergent markets like Brazil, China
and India, economic and business growth
opportunities are outpacing Europe and
America as significant investments are made
in local and cloud-based data infrastructures.
6. Advances in big data storage and processing
frameworks will help financial services firms
unlock the value of data in their operations
departments in order to help reduce the cost
of doing business and discover new arbitrage
In a recent research report, SunGard has identified opportunities.
ten trends shaping big data initiatives across all
7. Population of centralized data warehouse
segments of the financial services industry:
systems will require traditional ETL
1. Larger market data sets containing historical processes to be re-engineered with big data
data over longer time periods and increased frameworks to handle growing volumes of
granularity are required to feed predictive information.
models, forecasts and trading impacts
throughout the day.

3
www.inside-bigdata.com | 508-259-8570 | Kevin@insideBigData.com
 Big Data for Finance

Big data is an emergent trend driving

Retail Banking
Thanks in large part to the availability of data and
investments in enterprise analytics, and
the movement from in-person to online banking,
correspondingly, analytic excellence is today’s banking institutions look very different
central to much needed innovation in from those of just a decade ago. As the scale of
today’s financial services marketplace. data is overwhelming traditional systems, banks
must adapt to new technologies to unlock the
8. Predictive credit risk models that tap into power of their data.
large amounts of data consisting of historical
payment behavior are being adopted in Regulatory demands for banks have
consumer and commercial collections become much more stringent and
practices to help prioritize collections the increased level of historical
activities by determining the propensity for
delinquency or payment.
storage requirements have meant
9. Mobile applications and internet-connected
that banks must keep data for longer
devices such as tablets and smartphone are periods of time.
creating greater pressure on the ability of
technology infrastructures and networks to Banks have long been considered innovators when
consume, index and integrate structured and using data analytics to tackle numerous business
unstructured data from a variety of sources. challenges such as risk management, fraud
10. Big data initiatives are driving increased detection and price discovery. Yet today, the volume
demand for algorithms to process data, as of data is much bigger and more diverse than ever
well as emphasizing challenges around data before. At the same time, regulatory demands for
security and access control, and minimizing banks have become much more stringent and the
impact on existing systems. increased level of historical storage requirements
have meant that banks must keep data for longer
Big data is an emergent trend driving investments in periods of time. This complexity has necessitated
enterprise analytics, and correspondingly, analytic new approaches and technologies leading to big
excellence is central to much needed innovation data infrastructure.
in today’s financial services marketplace. Business
analytics applied to capital management, There are a number of motivating factors for
regulatory compliance, corporate performance, engaging big data technology in support of retail
trade execution, security, fraud management, banking:
and other instrumental disciplines is the principal • The availability and scale of data is extraordinary
innovation platform to improving strategic decision and requires a new technology mindset.
making. Analytics and the ability to efficiently and • A significant transition from in-person to
effectively exploit the big data technology stack, online banking resulting from the ease and
advanced statistical modeling, and predictive affordability of executing financial transactions.
analytics in support of real-time decision making
• A distinct need to collect and analyze this
across business channels and operations will
information in order to accurately assess risk
distinguish those companies that flourish in
and market trends.
uncertain markets from those that misstep.
• The availability of new data sources such as
The goal for this Guide is to provide direction for data from social media, blogs and other news
enterprise thought leaders on ways of leveraging feeds offer significant new opportunities. As
big data technologies in support of analytics with all online markets, banking is competitive
proficiencies designed to work more independently and banks are interested in using any
and effectively in today’s climate of working to opportunity, identified through data, to cross
increase the value of corporate data assets. sell and upsell customers.
4
www.inside-bigdata.com | 508-259-8570 | Kevin@insideBigData.com
 Big Data for Finance

Coupled with the above motivations, there

are a number of potential application
areas for big data in retail banking:
• Since banks weigh the risk of opening
new accounts versus the opportunity
to hold deposits, big data can screen
new account applications for risk
of default. In addition, big data can
identify high-risk borrowers for auto
loans.
• Maximizing customer touch points
leading to better customer service.
• Churn detection and avoidance.
• Cross-sell/up-sell using
recommender systems.
• Monetize anonymous banking
data in secondary markets.
Solving the challenging problem of
customer churn is one good example of Results of a global banking industry survey showing how banks see
how big data can make a real difference big data helping with business engagement.
for retail banking.
popular consumer destinations like Amazon or
“In 2012, over 50% of customers Yelp, customers have expectations about similar
either changed their bank or were experiences from the banking applications.
Without deep know-how about their customers,
planning to change.” banks may not be able to meet these expectations.
– Global Consumer Banking Survey The result? Lost revenue opportunities, low coupon
by Ernst & Young redemption rates, lower share of customer’s wallet
and lost competitive agility. In a nutshell, not being
Research shows that service quality is one of the able to gain insights from the goldmine of data
most prominent reasons why customers switch means banks are allowing their competitors to
banks. With credit card transactions, loyalty identify critical business trends and act on those
programs, and many other customer touch points, before they can, ultimately losing business.
banks possess far more data about customers than In summary, in order to advance the level of
any other industry, but in spite of all the data, customer intelligence banks must:
customer-centric companies like banks are often • Leverage big data to get a 360 degree view
unable to deliver effective personalized service. of each customer.
The main reason is the low level of customer
intelligence. • Drive revenues with one-to-one targeting
and personalized offers in real-time.
The world is increasingly interconnected, • Reduce business risk by leveraging predictive
instrumented and intelligent and in this new analytics for detecting fraud.
world the volume, velocity, and variety of data
being collected is unprecedented. As the amount • Achieve greater customer loyalty with
of data created about a consumer is growing, personalized retention offers.
the percentage of data that banks can process is • Employ the power of big data without worrying
going down fast. Based on their engagement with about complexities and steep learning curves.

5
www.inside-bigdata.com | 508-259-8570 | Kevin@insideBigData.com
 Big Data for Finance

There are many quality software tools allowing Kitenga is Hadoop-enabled for big data scalability
banking institutions to reap the benefits of big data. and allows for integration of disparate data sources
For example, the Kitenga Analytics Suite from Dell and cost efficient storage of growing data volumes.
is an industry leading big data search and analytics Kitenga can directly analyze Hadoop results using
platform designed to integrate information of all information visualization tools that bind directly
types into easily deployed visualizations. With this to Hadoop Distributed File System (HDFS) files, as
content mining and analytics solution, banks are well as index the created data and metadata into
a searchable form with embedded visualization
Kitenga also helps to reduce the capabilities. Kitenga acts as a wrapper of libraries
on top of Hadoop, with a drag-and-drop interface
pain of solving big data problems by to query, search, and analyze data directly against
providing out-of-the-box capabilities HDFS, without the need to create and script
for building search and analysis MapReduce jobs.
capabilities that work over big Kitenga enables financial institutions to integrate
data assets. structured transaction data with a variety of
unstructured private and public information
sources to quickly discern patterns associated with
able to transform complex and time consuming
fraud and immediately identify new transactions
manipulation of web-scale data resources into
that meet the model and require additional review
a fast and intuitive process. Banks can harvest
before being approved.
sentiments from Twitter feeds, blogs, news reports,
CRM systems, and other sources, and combine
them with demographic and regional data to better With the Dell Boomi master data
understand market traction and opportunities. management (MDM) solution,
Kitenga can be characterized with the following
high-level feature set:
you can take the complexity out
of integration. The software
• Analytics – Sophisticated natural language
processing (NLP), machine learning, predictive
solution offers on-premise and
modeling, sentiment analysis, social network cloud integration without
analysis, and visualization. appliances, software or coding
• Search – Interactive and intuitive. Search for easier transfers between CRM
interface allows business analyst to explore and back-office applications.
and exploit all data resources.
• Visualization – Interactive web-based authoring In the banking industry, it’s easier to provide
empowers business users to perform analysis, customers with a more personalized experience
visualize results and make decisions. when you have the latest data. But since so much
of that data lives in so many places throughout a
Kitenga also helps to reduce the pain of solving
business, it’s hard to make the best decisions quickly.
big data problems by providing out-of-the-box
With the Dell Boomi master data management
capabilities for building search and analysis
(MDM) solution, you can take the complexity
capabilities that work over big data assets. These
out of integration. The software solution offers
capabilities enable data analyst personnel to:
on-premise and cloud integration without
1. A
 uthor big data solutions by dragging-and- appliances, software or coding for easier transfers
dropping analysis components that operate between CRM and back-office applications.
over Hadoop for solving everyday problems.
2. E xecute analytical pipelines that operate over
Hadoop to scale up by scaling out
3. Monitor the jobs as they run

6
www.inside-bigdata.com | 508-259-8570 | Kevin@insideBigData.com
 Big Data for Finance

Credit Scoring C AS E ST U DY
Historically, loan and credit scoring methodology
employed by credit bureaus and used by banks and
other financial institutions has been based on a five
Credit Scoring at Novum Bank
component composite score including (i) past loan
by Marcel Wiedenbrugge
and credit applications, (ii) on time payments, (iii) Imagine you are active in the provisioning
types of loan and credit used, (iv) length of loan and of (micro) credit and a customer wants to
credit history and (v) credit capacity used. Until the borrow temporarily a few hundred euros
big data revolution, this approach has seen little from you. How do you determine whether
innovation in making scoring a commodity. it makes financial sense to do business with
this customer? For Joop Bruinzeel, Chief
Today, new technology platforms have opened
Credit Risk Officer (CCRO) at Novum Bank,
the doors for change in credit scoring and big data
this question is just another day’s issue. As a
scoring services are beginning to be available. Loan
provider of micro-credit, Novum Bank daily
and credit decisions are determined in seconds using
provides relatively small amounts (from €100
automated processes based on machine learning
to €600) to customers where traditional
algorithms. The breadth of data that can be used
banks have no interest due to a high risk
for credit scoring has expanded considerably. For
profile. Properly set up and tuned credit risk
each scoring decision, big data applications collect
management is essential.
data from a broad range of external data sources
ranging from social networks, e-commerce data, For assessing credit applications, Novum
economic databases, micro geographical statistics Bank recently started using STATISTICA, the
and other sources. In some cases, big data scoring analytical software solution from StatSoft
technology can use upward of 10,000 data points (now a part of Dell). In this interview I speak
in real-time to asses a customer’s creditworthiness. with Joop Bruinzeel about micro-credit, the
importance of credit scoring and the use of
As an extension to traditional scoring services, new analytical software.
technology companies using big data scoring are
providing scoring-as-a-service options for online Why did you choose STATISTICA?
loan and credit decisions. This type of solution is Joop: “Before I started working for Novum
provided to banks, debt collectors, e-commerce Bank, I immersed myself in modelling. I got
sites, leasing and other financial companies. These back in touch with another company that
systems can integrate into the customer’s existing was specializing in short-term loans and had
systems and/or website. (successfully) made use of STATISTICA. As the
shareholders of Novum Bank required mature
For a valuable use case example of how big data
risk management, STATISTICA was perceived
has transformed the credit scoring arena, see the
as a logical choice to go with.”
side bar “Credit Scoring at Novum Bank.” To assess
credit applications, Novum Bank in the Netherlands What do you like about STATISTICA?
recently started using the Dell STATISTICA, the Joop: “The beauty of STATISTICA is that you
analytical software solution. In the interview, Chief can build decisioning models which you can
Credit Risk Officer, Joop Bruinzeel talks about test on older portfolios (also called backlog
micro-credit, the importance of credit scoring and or backtesting). The workbench also offers
the use of analytical software. the possibility to add your own insights to the
models. That allows us to refine the models, so
that we can achieve better results.”
Read entire case study1

http://www.statsoft.com/Company/Press-Room/STATISTICA-News/EntryId/579/Credit-Scoring-at-Novum-Bank-Data-
1

Mining-defines-success-in-high-risk-lending
7
www.inside-bigdata.com | 508-259-8570 | Kevin@insideBigData.com
 Big Data for Finance

Back Trading & Testing

Another area of opportunity using big data The Dell™ | Cloudera™ Solution powered by Intel
technology is building back-testing software is a good option for addressing the needs of
solutions. Back-testing refers to the process of back-testing software. Cloudera Apache Hadoop
testing a trading strategy, investment strategy, Distribution (CDH) Enterprise consists of tools
or predictive model using existing historic data. for institutions that gather insights from vast
Back-testing is considered a special type of cross- data volumes and varied data types and find that
validation applied to time series data. The goal of managing large volumes of unstructured data
back-testing is to estimate the performance of a exceeds the capacity and capabilities of traditional
strategy as if it had been employed during a prior data intelligence systems. Aside from MapReduce,
period. This requires simulating past conditions financial industry firms can perform interactive
with sufficient detail, making one limitation of analysis on any data stored in Hadoop HDFS and
back-testing the need for detailed historical data. HBASE with Cloudera Impala.
Since major markets produce massive amounts of Intel has a long history with Hadoop — release 2.0
messages per day, US financial markets for example of its own Hadoop distribution in 2012. By aligning
produce around 50 billion data points per day, it is the Cloudera and Intel roadmaps in 2014, Intel
extremely computationally intensive task to process created the platform of choice for big data analytics
them. While back-testing is a computationally in order to help accelerate industry adoption of
intensive task it is also easy to parallelize. Multiple the Hadoop data platform and enable companies
trading days can be back-tested simultaneously to mine their data for insights that inform the
making it an ideal candidate for big data techniques business.
such as MapReduce. In order to complete back
testing within a reasonable time in an environment
like the mentioned example of US markets, big data
architectures like Hadoop are an invaluable tool.

Intel has played an important role in the Hadoop industry, culminating with its alignment with Cloudera in 2014

8
www.inside-bigdata.com | 508-259-8570 | Kevin@insideBigData.com
 Big Data for Finance

Adopting Big Data for Finance

The big data revolution has dramatically changed • It is important for banks, investment firms, and
the financial services industry. The following driving other financial services organizations to be
factors have motivated the need for financial sector able to collect and analyze this information in
companies to collect, store and analyze massive order to accurately assess risk and determine
volumes of data: market trends. This became apparent during
• Changes in the delivery of financial products the market downturn of 2007-2008, when
and services. Customers no longer need banks and brokerage houses scrambled
to physically visit their local bank to make to understand the implications of massive
deposits, make investment decisions or capital leverage and their ability to model
complete their banking transactions. Buyers and refine liquidity management. A single
and sellers of stocks execute their trades online bank might capture internal transactions
instead of relying on brokers. Individuals file exceeding two billion per month, in addition
their taxes using online tools versus meeting to collecting public data of over a billion
with a tax accountant to prepare and file. As monthly transactions. These tremendous
the industry has increasingly moved online, transaction volumes have made it nearly
it’s become faster, easier, and more affordable impossible to create models that take into
for consumers to handle their own banking account multi-year data sets using detailed
and financial transactions. data. Financial firms manage anywhere from
tens to thousands of petabytes of data, yet
• The result of these trends is that financial
most systems used today build models using
services and products have become
only samples as small as 100 gigabytes.
commoditized. Instead of establishing
Relying on data samples requires aggregations
relationships with a local service provider,
and assumptions, resulting in inaccuracies in
customers often choose the most convenient
projections, limited visibility into actual risk
and inexpensive online offering available.
exposure, instances of undetected fraud, and
Personal connections as a requirement for
poorer performance in the market.
customer loyalty has changed dramatically.
However, every action made by a customer • New sources of data. The increased pace
can be captured and analyzed by organizations of online activity has led to the availability
seeking to understand the behaviors and of new sources of data that are complex
preferences of their customers as they would to consume, such as unstructured social
have traditionally done through in-person media data. This information, if combined
interactions —now however, the process has with individual financial transactions and
become digitized. history, can help to paint a holistic picture
of individuals, families, organizations, and
• Increased volume of activity. The ease and
markets. Big data technologies are required
affordability of executing financial transactions
to process these structured and unstructured
via online mechanisms has led to ever-
data sources since using traditional relational
increasing activity and expansion into new
database and data warehousing technologies
markets. Individuals can make more trades,
for this purpose is no longer possible.
more often, across more types of accounts,
because they can do so with online tools in • Increased regulatory compliance. As result
the comfort of their own homes, or on the go of more rigorous regulatory compliance laws,
from a mobile device. Increased access and the financial services industry has had to store
ease of use translates into increased volume an increasing amount of historical data. New
of activity, which in turn translates into rapidly technology tools and strategies are needed to
growing data volumes. address these demands.

9
www.inside-bigdata.com | 508-259-8570 | Kevin@insideBigData.com
 Big Data for Finance

Not all industries are

created equal when it Exhibit 15
comes to being able to put Sectors differ in their ability to use and QUALITITAVE

data to use. Some, like the obtain value from bid data analytics Bubble size = GDP Competitive intensity to
adopt bid data
construction industry, are Sectors studied
Highest Moderate
in this report
constrained by the amount Big data ease of capture
High Low
Reflects ability to own or access data and analytics
of data they can capture Higher
and even more hamstrung
Utilities Natural
by their ability to get value Infor-
resources mation
from it. Others, like the Health-
care Finance and
finance industry, both Manufacturing providers insurance
generate a lot of data and Transportation
can put it to use. In the and warehousing
Professional
graphic below from the services
Real estate
U.S. Bureau of Economic Management and rental
Construction of companies
Analysis, it is clear that
the finance industry ranks Administration, support,
highest in terms of its and waste management Accommo-
dation Government
Retail
ability to use and obtain and food and trade
value from big data. Educational services
Lower Arts and entertainment
Determining the appro- Higher
Lower
priate level of engagement Big data value potential
for a big data deployment Reflects value of data and/or competitive advantage achieved
project is an important
Source – US Bureau of Economic Analysis; McKinsey Global Institute analysis
consideration toward insur-
ing the success of the Finance industry is highest ranking in terms of its ability to use and obtain value from big data.
project. For example,
department-level big data projects generally are • To offset flat or declining revenue streams,
more successful than large-scale initiatives which financial services firms need to develop new
routinely fail. An incremental approach is better. big data centric products while also targeting
existing products to new audiences.
Here is a short list of guidelines toward the adoption
of big data for finance: • Use big data and its associated tools not only to
identify risks in real time and improve forensic
• Develop methods and services for the
accounting abilities but also to evaluate the
valuation of data — and extend their role in
risks and rewards of long-term investment in
compliance and internal control to the ethical
new products and new markets.
and effective stewardship of data assets.
• Gaining agility starts with an assessment of
• Unite disparate data from a variety of systems
existing processes and systems. Financial
designed to meet the diversity across regions
industry firms must identify what existing
regarding language, regulations, currency,
practices will not support the progress they
time zone, etc.
need. To get significant advantage in today’s
• Use big data to offer more specialized decision- competitive landscape, they should pursue
making support — often in real time— and technologies that support new, innovative
decide when data can most usefully be shared practices.
with internal and external stakeholders and
monetized as new products or services.

10
www.inside-bigdata.com | 508-259-8570 | Kevin@insideBigData.com
 Big Data for Finance

By implementing a Hadoop solution, an organization

Spark is an open-source data processing approximately 20 billion events per day
analytics cluster computing is able to significantly decrease operational costs:
decreased storage costs from ~$17/GB to ~21
framework built on top of HDFS.
cents, experience a one year payback on the entire
Spark serves as evidence of the initiative.
continuing evolution within the
The Intel Powered Dell™ | Cloudera™ Apache
Hadoop community— away from Hadoop Solution is a superb choice for lowering
being a batch processing framework the barrier to adoption for financial institutions
tied to the two-stage MapReduce intending to use Apache™ Hadoop® in production.
paradigm to a more advanced Dell Xeon based PowerEdge servers, Force10
networking, the linux operating system and the
in-memory, real-time platform. Cloudera Manager tools make up the foundation
on which the Hadoop software stack runs.
Hadoop represents a good path for financial sector You can start planning out your Hadoop
firms to adopt big data. With Hadoop, firms have infrastructure using one of the global network of
access to a powerful platform providing both highly Dell Solution Centers, technical labs that enable
scalable and low cost data storage tightly integrated you to architect, create a proof-of-concept
with scalable processing. Financial firms are now
able to tackle increasingly complex problems by
Using StackIQ’s Cluster Manager
unlocking the power of their data. The capability to
understand and act upon their data opens the door software the customer was
to a richer and more robust financial ecosystem. able to rapidly provision and re-
A recent addition to the Hadoop ecosystem is provision the servers to run the two
Spark. Spark is an open-source data analytics different applications with various
cluster computing framework built on top of configurations. The use of Cluster
HDFS. Spark serves as evidence of the continuing Manager enabled the customer to
evolution within the Hadoop community — away
from being a batch processing framework tied to
complete more and higher quality
the two-stage MapReduce paradigm to a more tests than originally expected.
advanced in-memory, real-time platform.
(POC), validate, and build data center solutions.
Now, these firms can better serve their customers, In a recent POC for a large provider of financial
understand their risk exposure and reduce incidents data, Dell partnered with StackIQ to configure a
of fraud. Here is a short list of benefits afforded by 60-datanode cluster to compare two different big
Hadoop to the financial services industry: data technologies, Cassandra and HBase. Using
• Scalability to capture and analyze data StackIQ’s Cluster Manager software the customer
previously untapped was able to rapidly provision and re-provision the
• An economical way to store and process data servers to run the two different applications with
various configurations. The use of Cluster Manager
• A central repository for various data
enabled the customer to complete more and higher
structures from existing and new data sources
quality tests than originally expected.
• Operational efficiency by moving jobs to
technology designed to process multiple
data types
• The ability to ask different questions to
improve decision making

11
www.inside-bigdata.com | 508-259-8570 | Kevin@insideBigData.com
 Big Data for Finance

Security Considerations
An important application area where big data is Customers question their trust in their bank’s ability
taking a firm foothold in many financial industry to provide security and protect their privacy. Hard-
firms is information security. In conjunction with earned customer loyalty diminishes. How can you
the traditional 3 Vs of big data, financial industry detect fraud and stop attackers before they threaten
firms must consider a fourth V: vulnerability. To your financial institution and its customers? Big
manage big data effectively, you must keep it secure data technologies can help by enabling financial
and compliant with regulatory requirements at all firms to not only capture in near real time every
times (vulnerability). Protecting a vast and growing event that occurs across the entire organization but
volume of critical information — and being able to also provide context to understand these events so
search and analyze it to detect potential threats — is information can be shared to better issue alerts of
more essential than ever. As the software platforms potential and actual threats.
(e.g. Hadoop) supporting this quantity of data
Many finance industry firms are using big data to
move to mainstream use, managing their security
detect and/or prevent fraud. Big data supports
and availability becomes a big data challenge in
what’s known as continuous or behavioral
and of itself, requiring continuous diagnostics and
authentication, a process that can help prevent
monitoring.
fraud. Further, detecting security breaches
using huge volumes of security data along with
The increasingly global nature of the unstructured social media data, combined with
financial services industry makes new big data tools such as Hadoop, enables
it necessary to comprehensively financial industry firms to be more proactive about
security. Big data can enhance data security for the
address international data security finance industry through:
and privacy regulations. • Understanding activity patterns among
customers and the broader industry.
Banking and financial institutions need to secure the
• Sharing of data – critical especially about
storage, transit and use of corporate and personal
emerging attack vectors and threats.
data across business applications, including
online banking and electronic communications of • Increasing reliance on data to predict
sensitive information and documents. The typical attacks, based on trends that are targeting
IT environment consists of a mix of new and legacy the industry.
systems and applications across highly distributed One particularly good solution for finance industry
networks of branch offices, call centers and web data security requirements is Dell’s SharePlex
portals. Many of the traditional point security Connector for Hadoop. Proactive security requires
solutions that are deployed add complexity and data analytics for a business intelligence advantage
management costs, and leave gaps between and essential decision-making insight. The Hadoop
systems and applications that are highly vulnerable framework gives you that, but integrating data can
to attack. The increasingly global nature of the be time consuming, providing only snapshots that
financial services industry makes it necessary quickly become out of date. SharePlex Connector
to comprehensively address international data for Hadoop loads and continuously replicates
security and privacy regulations. changes from an Oracle database to a Hadoop
Financial institutions are top targets of cybercrime. cluster— in near real time to Hive and HDFS, and
While all types of businesses are vulnerable to in real time to Hbase. This gives you all the benefits
attacks by criminals, it’s the security breaches at of maintaining a real-time or near real-time copy of
financial firms that elicit the most media attention, source tables, so your organization can efficiently
public scrutiny and legislator consternation. When and cost-effectively perform big data analytics in
threats occur, it’s more than financial loss at stake. support of enterprise security.

12
www.inside-bigdata.com | 508-259-8570 | Kevin@insideBigData.com
 Big Data for Finance

Regulatory Compliance Considerations

The regulatory environment under which banks regulatory environment is a matter of maintaining
operate adds substantial operational concern for an efficient and large-scale data management
the business. Stringent regulatory compliance laws infrastructure. This level of compliance can be
have been put in place to improve operational addressed via big data technologies.
The wide reaching effects of the economic
Financial industry companies meltdown in 2008 were largely due to the lack of
have recognized that the key to visibility into consumer actions and groups with
optimizing their business operations related risk profiles, combined with the increased
flow of consumer funds into financial firms. Financial
in today’s regulatory environment is industry firms, as well as the underlying economy,
a matter of maintaining an efficient were more susceptible than statistical models had
and large-scale data management predicted. As a result, the finance world set out
infrastructure. to find new classes of technologies that would
allow them to manage and take advantage of ever-
growing data sets. In the process, they evaluated
transparency. With government regulation driving
the technologies deployed by large web concerns
many business processes, banks have apprehension
like Facebook and Google. In short, they found
leading up to a national election because they don’t
Hadoop: an open source software architecture
know what direction the regulatory wind might
that enables distributed parallel processing of huge
blow. Currently, financial services organizations
amounts of data across inexpensive, commodity
are held much more accountable for their actions,
servers. With Hadoop, no data set is too big or
and are required to be able to access years of
complex.
historical data in response to regulators’ requests
for information at any given time, specifically:
• The Dodd-Frank Act requires firms to maintain Many of today’s top financial
records for at least five years. services firms have already deployed
• Basel guidelines mandate retention of risk Hadoop to create centralized data-
and transaction data for three to five years. hubs combining large volumes of
• Sarbanes-Oxley requires firms to maintain diverse and detailed data, crafting
audit work papers and required information competitive advantage in several
for at least seven years.
key business application areas.
• FINRA/Tradeworx Project – SEC requires the
creation of a real-time transaction monitoring
system to detect potentially disruptive market Hadoop is a stable and dependable platform that
activity stemming from high-frequency financial organizations can trust for their big data
trading. A consolidated audit trail or CAT (for requirements. Using big data solutions driven
day after monitoring) by FINRA is to include by Hadoop allows financial services firms to
orders, quotes, updates and cancellations, optimize capital leverage while maintaining the
plus a real-time system by Tradeworx are two reserves required by regulators. Many of today’s
initiatives for market surveillance. top financial services firms have already deployed
Hadoop to create centralized data-hubs combining
Further, these records must be available on
large volumes of diverse and detailed data, crafting
demand, or in some cases must be normalized
competitive advantage in several key business
and sent to regulators proactively. Financial
application areas.
industry companies have recognized that the key
to optimizing their business operations in today’s

13
www.inside-bigdata.com | 508-259-8570 | Kevin@insideBigData.com
 Big Data for Finance

The Hadoop software stack is a good choice to

unlock the power of the financial firm’s data One solution to the rigors of
assets including but not limited to compliance regulatory compliance is to employ
requirements:
the Dell SecureWorks Security & Risk
• Modeling market risk – using detailed data Consulting team designed to provide
to give banks better insight into the material
behavior of complex financial instruments the expertise and analysis to help you
and creating more predictable results from enhance your IT security posture,
these investment vehicles. Using Hadoop, reduce your information security risk,
banks can build and evaluate models in an facilitate compliance and improve
expedient manner, leading to a high degree of
competitive advantage.
your operational efficiency.
• Modeling consumer risk – giving greater
insight into capital availability and liquidity. One solution to the rigors of regulatory compliance
Using Hadoop-based solution, financial is to employ the Dell SecureWorks Security & Risk
services organizations are able to optimize Consulting team designed to provide the expertise
capital leverage while maintaining the and analysis to help you enhance your IT security
reserves required by regulators. posture, reduce your information security risk,
• Fraud detection and AML capabilities – facilitate compliance and improve your operational
allowing banks to detect critical breaches efficiency. The Dell team has helped thousands of
faster, saving money that directly affects the customers design their strategic security programs,
bottom line. As financial firms roll out new assess and test their defenses, resolve critical
products and services to customers, the information security breaches, and meet their
committers of fraud and money laundering compliance mandates.
activities constantly are adapting to fraud Dell SecureWorks’ Compliance and Certification
prevention techniques. services offer assessments specifically tailored
• Recommendation/personalization – providing to address regulatory issues for financial services
sophisticated tools for cross-sell and up-sell firms. Security consultants work with you to
purposes so banks can offer new financial gauge the current risks to your mission critical
products and services to their existing high- IT assets, and to assess the current state of your
value customers. With the ability to track security posture as compared to best practices and
user interactions across applications and regulatory guidance.
marketing channels in real time, banks
can dynamically categorize customers and
experiment with selectively targeted offers,
including investment strategies and savings
advice to help customers improve their
financial discipline. The result is an improved
and more personalized client experience.
• Mortgage portfolio valuation – identifying
which mortgages are candidates for default.
The Hadoop platform engages the use of
unstructured data sources that offer greater
visibility into mortgage models, such as
property valuation sites, local consumer
offers such as crime report data and public
sentiment data.

14
www.inside-bigdata.com | 508-259-8570 | Kevin@insideBigData.com