Archive for the ‘Data Services’ Category

Data Strategy Webinar Series

Maximizing Data to Drive Your Organization Forward 

Get a jump-start on your 2018 professional development. We invite you to join subject matter experts from Protiviti’s Data Management & Advanced Analytics team for this informative five-part webinar series. Join us on Thursdays beginning February 1 and continuing through March 8 at 2:00 p.m. eastern to learn what’s new in Business Intelligence and Data Governance.

Register for one or all five!

Building a Comprehensive Data Strategy
Speaker: Jeremy Stierwalt, Director
Thursday, February 1 @ 2:00 p.m.

Data is arguably the most important asset a company possesses, and a data strategy allows the company to recognize that its data is a structured, comprehensive, cross-domain asset. The organization’s volume of data, where its data is stored, who in the organization is responsible for the data and where future data investments will be made are all components of this comprehensive strategic approach. Jeremy will analyze the anatomy of a data strategy and will also share success stories from Protiviti clients.

Key Learning Points:
– Learn what to include when defining the scope of a data strategy
– Discover how to develop a road map for the organization by defining key gap-closing initiatives
– Recognize how strong, experienced leaders can ensure success
– See what a Future State recommendation would look like
– Learn how to formalize a Data Governance process as part of the strategy development

Jeremy Stierwalt brings more than 20 years’ experience in all aspect of data management and governance, business intelligence, advanced and predictive analytics solutions. This includes vertical expertise in Financial Services and Insurance, Manufacturing, Consumer Products, Retail, Sports and Entertainment, and Professional Services coupled with Line-of-Business expertise in Finance, HR, Supply Chain, Sales and Marketing, and Procurement functions. Jeremy has held various leadership positions including membership to the Executive Leadership Team, and is a frequent speaker at National & Regional Conferences on topics related to data governance & management, big data, and analytical solutions.

6 Simple Steps to Solve Your BI User Adoption Challenges
Speaker: Steve Freeman, Managing Director
Thursday, February 8 @ 2:00 p.m.

In spite of significant Business Intelligence (BI) investments, just a relatively small percentage of companies consider their BI programs a success. Many continue to struggle with how to best adapt to changes in the business and evolving systems needs. Join Steve as he walks through the six components that make up the FRA2ME methodology, which allows every client to maximize their BI investment, turning business users into advocates and dramatically growing BI adoption throughout their organizations.

Key Learning Points:
– Explore the six steps to building FRA2ME methodology in your organization
– Learn how FRA2ME helps organizations delineate between information and insights
– Understand how a “Build, Adapt, Outreach” approach helps users adjust to change
– Discover how to have your end users saying “I don’t know how I’d do my job without this”

Steve Freeman, a Managing Director in Protiviti’s Data Management and Advanced Analytics practice, developed the FRA2ME methodology to help clients generate “raving fans” among end users. Responsible for Protiviti’s SAP Analytics practice, Steve also serves on the firm’s Financial Services practice leadership team. He has held numerous roles in sales and executive management in the Business Intelligence and Analytics space including: SAP BusinessObjects, Oracle, Verint and Cognos. A thought leader in analytics and end-user adoption, Steve’s expertise also centers on Customer Insight Analytics, Sales & Financial Forecasting, and Organizational Optimization.

Managing the New Currency: Developing a High-Performing Data Management Organization
Speaker: Don Loden, Managing Director
Thursday, February 15 @ 2:00 p.m.

Data has become so critical for success at any organization that it’s now being described as the new currency – impossible for a business to grow and thrive without it. Attendees will learn what successful companies are achieving with strong data management in place and will learn how to avoid common pitfalls that jeopardize solution adoption. Don will also outline why quality data and a trusted data platform are key components for both data and analytics success.

Key Learning Points:
– Learn why many data management efforts fail and how to align those efforts with the business
– See how to follow the crumbs to find the reward of user adoption
– Learn why a trusted data platform is critical for success

Don Loden is a thought leader in the data warehousing and governance space. With more than 15 years’ experience in ETL Architecture, Development, and Tuning: Data Modeling; Data Warehouse Design: Analytics Architectures; Business Analysis and more, he also speaks regularly at industry functions and has written numerous articles for publications focused on data warehousing, master data management, governance, SAP HANA, and data quality best practices. Don is also the author of two books on SAP technologies and strategic use cases. His sales acumen coupled with his expertise has helped the Data and Analysis solution grow substantially over the last several years.

Avoid an Epic Fail: Why Data Governance is a “Must Do”
Speakers: Josh Hewitt, Director
Thursday, February 22 @ 2:00 p.m.

Do you trust the data your organization uses for strategic, financial or regulatory reporting? Is there clear ownership and understanding of key data elements? Does identifying root causes of data quality issues often leave you scratching your head and scrambling for help? Data Governance needs to be viewed as a Program and not a one-time Project. Discover real-life lessons learned from our clients at varying levels of Data Governance maturity and will provide tips to ensure you can best prepare and improve your Data Governance capabilities along your journey.

Key Learning Points:
– Understand scoping and key functions of Data Governance programs
– Review different operating models
– Examine components of achieving higher maturity within people, processes and technology
– Learn how to build a business case and drive value through Governance
– Explore how Data Governance supports an overall Data Strategy and Advanced Analytics

Josh Hewitt, a Director, has more than 13 years’ experience in Financial Services and has a wealth of knowledge surrounding data governance/management, information technology, regulatory/compliance Risk Management and program/project management. Having been part of multiple data governance program planning and implementations, Josh offers a unique perspective on what has worked in other organizations and the best approach to the implementation of successful programs. He also has significant experience in data governance/quality tool selection and evaluation and a strong understanding of the benefits and pitfalls of many different solutions and can help Financial institutions evaluate, select and implement solutions.

Managing Data Governance in a Cloud-Focused World
Speaker: Narjit Aujla, Manager
Thursday, March 8 @ 2:00 p.m.

Companies like Amazon and Microsoft are opening the door for anyone with internet access to stand up rogue environments outside of corporate guideline. This produces a unique set of challenges from a Data Governance standpoint like standardizing KPI definitions, Master Data Management, and Data Security. Narjit will take attendees through the steps necessary to integrate the cloud movement into a Data Governance approach, while still enabling the usefulness and practicality of cloud applications.

Key Learning Points:
– Learn how metrics used by cloud shadow applications adhere to the Enterprise Data Dictionary
– Learn how to assess the risk of a cloud service to better protect its sensitive data
– Examine how IT manages authentication and data flow within a cloud application
– Learn how to create concrete and reasonable cloud data governance controls
– Understand what steps to take to ensure the cloud provider adheres to vendor requirements

As a manager in Protiviti’s Data Management and Advanced Analytics practice, Narjit Aujla has worked with companies spanning a variety of industries. Narjit specializes in Data and Analytics with a focus on Enterprise Information Management (EIM), front-end dashboard development, data modeling, and architecture strategy. He also works closely with SAP BusinessObjects Enterprise suite of tools including SAP Design Studio and SAP Web Intelligence in addition to other analytical tools such as SAP HANA Studio and SAP Lumira, Narjit has experience working with various database solutions, including Oracle DB2 and Microsoft SQL Server. He has also helped businesses refine their Data Governance strategies, identifying gaps in business process using data profiling tools such as SAP Information Steward.

Data Quality Improvement with SAP Data Services

First things first, what is data quality and why should I care?

Data quality is crucial to operational and transactional processes and to the reliability of business analytics and business intelligence reporting. You can’t make good decisions with bad data. Data quality is essentially ensuring that all of your data coming in and all of your loaded data is of high quality.

Well, what is “high quality?”

High quality data is:

  • Accurate
  • Complete
  • Up to date
  • Relevant
  • Consistent across data sources
  • Reliable
  • Accessible

So what happens when you realize your data doesn’t meet these criteria? How do you get started when you want to implement a data quality initiative? The first two steps are what this blog is about: preparing your data and standardizing your data.

Preparing Your Data


Ideally, you start with a Data Prep Phase. This is the process of collecting, cleaning, and consolidating your data into one place for use. Gartner estimates that up to 80% of the work in data analytics is done during the prep phase! So this isn’t an area to downplay the importance of.

Questions to ask yourself during this phase:

  • Where is my data? Where does it live? What is the data source?
  • Who uses the data? There will be stakeholders in varying business and functional areas to consider and involve; be sure to seek experts who not only understand the data, but also the business processes.
  • Is the data any good? Is it usable?
  • What is the best way to consolidate the data?

Standardizing Your Data


Once you’ve completed the prep phase, you’re ready to move on to the Data Cleansing and Standardizing Phase.

Data standardization is the next step to ensuring that your data is shareable across the enterprise. You want to make sure your data is the same across the organization. If not, sales figures may not match up, your detail reports may not confirm your summary reports, addresses will not be valid. These types of situations result in wasted time, additional overhead, bad decisions and a lot of frustration.

SAP Data Services is a great tool for getting to “one version of the truth.” SAP Data Services:

  • Cleanses and standardizes customer data such as name, addresses, emails, phone numbers, and dates; prevents incorrect data such as invalid contact information
  • Manages international data for over 190 countries and reads and writes Unicode data
  • Removes errors to uncover the true content of the database
  • Improves integrity of data to identify matches
  • Ultimately creates a single customer view

SAP Data Services can also help you apply and enforce data quality rules whenever data is created, updated, or moved. It also allows you to perform data quality checks anytime, in real-time, on data sets before analyzing, moving, or integrating data.

SAP Data Services helps your organization move toward that “one version of the truth” and stave off hours of wasted time and rehashed problems. Your departments will have the same definitions and terms to work with, correct data and clean information.

Standardization is the cornerstone of business intelligence.

For more information and to see some examples of how SAP Data Services transforms data, you can listen to a pre-recorded webinar I gave called “Expert Guidelines for Building a Data Quality Assessment & Improvement Initiative.”

Expert Guidelines for Building a Data Quality Assessment and Improvement Initiative

You can also read more about Data Quality and other SAP Resources in my other blog series:

Getting Started with SAP BusinessObjects Data Quality

Bruce Labbate HeadshotAbout Bruce Labbate
Bruce is a business intelligence consultant specializing in data warehousing, data quality, and ETL development. Bruce delivers customized SAP Data Services solutions for customers across all industries. With Decision First Technologies, A Protiviti Enterprise, Bruce utilizes Data Integrator, Data Quality, Information Design Tool, and a variety of database technologies, including SQL Server, DB2, Oracle, Netezza, and HANA.


Categories: Data Services

Text Analysis with SAP

There’s a lot of value in unstructured data, but parsing it isn’t something any old analytics engine can do. In this E-Bite, find out how to use SAP HANA, SAP Data Services, and SAP Predictive Analytics for linguistic and sentiment analysis. Get a crash course in the fundamentals of text analysis, and then learn how to perform full-text indexing, text mining, entity extraction, and more. Do you know what your customers are saying?

  • Perform linguistic and sentiment analysis with SAP HANA
  • Explore SAP Data Services’ ready-to-use text analysis capabilities
  • Use real-life sample data to parse customer reviews and other unstructured data

Checkout this new E-Bite from Hillary Bliss and Bruce Labbate! Now available for pre-order!

Hilary BlissAbout Hillary Bliss
Hillary Bliss is the analytics practice lead at Decision First Technologies and specializes in data warehouse design, ETL development, statistical analyis, and predictive modeling.



Bruce Labbate HeadshotAbout Bruce Labbate
Bruce Labbate is a principal consultant at Decision First Technologies for business intelligence, data modeling, and data architecture.

Maximizing SAP Data Services

It really takes convincing and in-depth stakeholder understanding to get buy in for new IT projects. It is equally difficult to get time from business users for any IT implementation, let alone a simple SAP upgrade, that seemingly has little return from a business perspective. This explains the reason many corporations are not upgrading their SAP system or even leveraging the existing SAP BI tools they’ve already acquired!

I was recently in a client demo with Don Loden and Roy Wells, listening to the client ask some questions about SAP Data Services and other EIM tools. We noticed the client had purchased Data Services with the BusinessObjects toolset, but they were not using the software and were having terrible performance issues with their old ETL tool. I believe there are a lot of SAP customers who are in the same situation, and who are at a crossroad about whether to invest more in a new tool or stick with the old, problematic one. Luckily, most organizations that use BusinessObjects for reporting either already have Data Services, or can easily add it to their existing license to resolve most of their current ETL issues.

For example, another client recently implemented BW on HANA. They invested a great amount of time and resources into creating well-designed DSOs, Cubes, multiproviders, as well as incorporating data from NON-SAP sources. Since they already had Data Services, there was no need for new development and modeling efforts. We were able to stage and transform the external data, SAP BW DSOs and Cubes into SAP HANA, allowing us to directly create views for their reports. Most of our look-ups were done in Data Services, and we were able to leverage ECC tables and extractors, providing increased performance.

SAP Data Services is still the best-of-breed ETL tool that delivers a single-enterprise solution for data integration, data quality, data profiling, and text data processing. It allows you to integrate, transform, improve, and deliver trusted data to critical business processes. There is so much that you can accomplish with it.

Whether you have BW, BPC, HANA or just use BusinessObjects for reporting, often your data extraction can take hours and reports can be delayed. With Data Services, you can get significant data load improvement and improve the quality of your data and reports, all delivered in a timely manner.

EricAbout Eric Vanderpuije
Eric Vanderpuije is a Business Intelligence Consultant with an extensive background in BW, data analysis, and ETL development.  Eric has over 10+ years of experience of which 6 + focusing in BI Architecture, Data Modeling, BW tools set including developing Infocubes, DSOs, Multi provider, DTP, Transformation, Data sources, BW Reporting BEx Analyzer, InfoSet Query, and Web Reporting (WAD), BI Integration, and Datasource enhancement.

Eric came to Decision First Technologies after being with Wipro Technologies, where he was a BW lead. He also has experience in data warehousing with SAP DataServices, focusing on data integration and warehousing with SAP DataServices and a variety of database systems, including HANA and Netezza.

Categories: Data Services

SAP Real-Time Enterprise Brings Real-Time Problems

To adapt to the challenges of the SAP real-time enterprise, organizations have to shift from using ‘after the fact’ latency processes to address data quality to implementing those that take place at the point of data entry.

Governing data and measuring and monitoring data quality have always been important to companies, as a result, they spend lots of time and money governing data quality and processes.  Data quality in the source system is more important than ever when a new technology like SAP HANA enters the picture.

Click here to read full article


contributor_don_loden_lg About Don Loden

Don is a principal consultant with full life-cycle data warehouse and information governance experience in  multiple verticals. He is an SAP-certified application associate on SAP Data Services, and is very active in the  SAP community, speaking globally at numerous conferences and events.  He has more than 14 years of i  information technology experience in the following areas: ETL architecture, development and tuning, logical  and physical data modeling; and mentoring on data warehouse, data quality, information governance and ETL  concepts.

Categories: Data Services, HANA

Intro to Data Services Workbench

Since the release of SAP Data Services 4.0, the design team has been floating the idea that the Data Services Designer would be replaced as the main design interface for SAP Data Services in the near future.  Perhaps this is due to the thick-client installation that requires an unbroken connection to the repository database, which may find difficult to work with as telecommuting becomes so much more prevalent. With the release of Data Services 4.1, the Data Services Workbench was introduced as an automatic install whenever the Designer client was installed.  Initially, the workbench really performed only the most basic function: replicating data from a source database or datastore to HANA or Sybase IQ databases. Since then, we have seen the functionality of the Workbench expand to support most applications and databases as sources and targets, and to incorporate additional functionality to develop new content within the Workbench interface. This blog post will review the progress towards full functionality and note some new features and differences between Data Services Designer and Workbench.

Replication Jobs

The first purpose of Workbench was to provide a quick replication tool to port data from other database systems to SAP HANA. Creating a new replication job opens up an interface that simply shows the tables being replicated with no dataflow-type structures.

data services workbench


In the properties window below for each table, the user can change settings like basic column mappings, adding filters, and setting the basis for delta loads, but there are no complex operations supported, like joins, or anything other than a single simple query transform.

To download full PDF and Continue Reading…


Hilary Bliss

About Hillary Bliss

Hillary is a Senior ETL consultant at Decision First Technologies, and specializes in data warehouse design, ETL development, statistical analysis, and predictive modeling. She works with clients and vendors to integrate business analysis and predictive modeling solutions into the organizational data warehouse and business intelligence environments based on their specific operational and strategic business needs.  She has a master’s degree in statistics and an MBA from Georgia Tech.


Categories: Data Services

Defining Aliases Within Your Datastore in Data Services

When developing code within SAP’s Data Services enterprise information management tool, generally requirements dictate having to extract and load data across multiple environments.  Data services utilizes datastores to allow you to connect to a variety of different data sources such as a web service, an adapter, or more commonly a database.  Those database data sources can include everything from SQL Server, Oracle, DB2, even Netezza.  This blog will describe setting up datastores against a Netezza database to utilize the alias functionality to simplify the process of migrating code from development to production.

What is Netezza and how does Data Services connect to it?

Netezza is a powerful Data Warehousing appliance that integrates the database, server, and the storage components into a single system. Netezza is purpose-built for data warehousing and is specifically  designed for running complex data warehousing workloads. As a result of using proprietary architecture to deliver fast query performance, minimal tuning and administration, and high scalability; Netezza is an ideal database system to use for your data warehousing needs.  As with any relational database system, Data Services can easily connect to Netezza using a datastore connection. Data Services can then import the metadata that describes the data through that connection. If the metadata is identical across multiple environments you can have multiple data configurations within one datastore.


Having multiple configurations within one datastore eliminates the need to create a datastore for every single database you need to connect to; which can speed up development time and prevents unnecessary clutter in your local object library.


To download full PDF and Continue Reading…

shaun2About Shaun Scott

Shaun Scott is a Business Intelligence consultant specializing in data warehousing and ETL development. Shaun delivers customized SAP BusinessObjects solutions for customers across all industries. With Decision First Technologies, Shaun utilizes SAP Data Services, Universe Design, Web Intelligence, and a variety of database technologies, including SQL server, DB2, and Netezza.


Categories: Data Services

Compressing Snapshots in Data Services

Sometimes, when capturing data in a data warehouse, we need to store time-variant pieces of data about a transaction. This somewhat blurs the lines between a traditional fact table and a dimension, since in the traditional model, time-variance is mainly the domain of a dimension.

Take the example of a production backlog at a manufacturer. When an order is made, particularly in an organization that manufactures large and/or complex goods, it may take some time to fulfill. Maintaining a consistent backlog is also a key to ensuring consistent production planning that’s not beset with shutdowns, inefficiencies, or missed delivery dates.

Keeping a backlog at a granular level generally requires tracking backlog on an order-by-order basis. That way, anything about an order (that’s in your warehouse) can be analyzed to look for trends in the business. There’s just one issue: keeping a snapshot of every order in backlog for the full amount of time it’s in backlog can take up a lot of space. For instance in a mid-size company: if the average order is in backlog for three months and the company receives 10,000 orders per year, that’s nearly a million records per year in a daily snapshot. After a while, that can really add up. It’s no wonder Bill Inmon said, “The management of these every day, ongoing changes can consume an enormous amount of resources and can be very, very complex. It is this type of load that rivets the attention of the data architect.” (Snapshots in the Data Warehouse, pg. 2, white paper at


This example could also apply to general A/R snapshots by account, though in many organizations, this snapshot is taken on a monthly basis, so the problem is less imperative.

As the snapshot grows, a simplified version may look something like this:

Order Key Date Key Item Key Past Due Backlog Amount Future Backlog Amount
50 20131201 76 0.00 65,000.00
50 20131202 – 20131231
50 20140101 76 35,000.00 30,000.00
50 20140102 76 0.00 30,000.00
50 20140103 – 20140114 .. .. ..

In this example, an order for $65,000.00 was placed on December 1, 2013. $35,000.00 worth of product is for delivery on December 31, while $30,000.00 worth of product is scheduled for later. So, all $65,000 goes into future backlog when the order is received.

To download PDF and Continue Reading…

Britton HeadshotAbout Britton Gray
Britton has been working in software development and Business Intelligence for the past fourteen years, and has been working with SAP BusinessObjects reporting, analytics, and information management tools for six years. He is certified in Data Integrator and Web Intelligence, and enjoys developing end-to-end BI solutions that enable customers to gain new insights into their data.

Categories: Data Services

Integrating Secure FTP into Data Services

SAP Data Services includes decently robust native support for FTP transport, as long as you don’t mind it being non-secured. However, understandably, many applications require the use of secured FTP transport methodologies, since the data being transmitted is often of a sensitive nature. You probably don’t want to risk your data being intercepted in transit, especially if its going outside of the organization’s firewall. Fortunately, programs like WinSCP are available, and are robust, scriptable, and free. We can then integrate the scripting into Data Services to create a relatively seamless solution.

What You’ll Need: 

  • WinSCP (available at
  • Data Services
  • A text editor
  • FTP credentials and protocol information
  • Known directory structures on both source and target systems
  • A little bit of time

In this example, we’ll copy over a remote directory’s contents to a local directory.

Step 1: Install WinSCP
Download WinSCP at You can run a pretty standard install.

  • Make a note of the installation directory
  • Set the program to use .ini configuration, instead of the default, which is to write configuration to the registry. I’ll explain why (and how to fix it if you already goofed) below.

Step 2: Make Sure You’re Using .ini Configuration
WinSCP stores remote server keys in its configuration area. Generally, the first time you connect to a remote server, you need to accept the remote key. After this, it’s stored in that area for any future uses. Since we want all of this scripted, we want to make sure there is no user input required. If there is, the entire process will fail, since Data Services will interpret the prompt as an execution failure.

To download PDF and Continue Reading…

Britton HeadshotAbout Britton Gray
Britton has been working in software development and Business Intelligence for the past fourteen years, and has been working with SAP BusinessObjects reporting, analytics, and information management tools for six years. He is certified in Data Integrator and Web Intelligence, and enjoys developing end-to-end BI solutions that enable customers to gain new insights into their data.

Categories: Data Services

Extracting Metadata from ODBC Sources

Metadata is essential to well documented ETL processes. Ideally, this metadata starts with the source system table and column descriptions and other system specific information. When this information is not available directly inside of Data Services, retrieving it manually can fill in the gaps and help your users get a complete map of how data arrives in the warehouse.

Generally, the lack of metadata is caused by the absence of a native driver to the source database. For example, using ODBC to access a DB2 source on an AS400, the ideal solutions is to use DB2 Connect, a native driver, but that option can be price prohibitive for some clients. Additionally, some third-party vendors only allow connectivity via ODBC.

How do you go about getting the metadata? Most databases have system tables that contain the data we are seeking. A quick Google search for your source database and platform should provide you the system table name(s) you need. For this version of DB2, the table is called SYSIBM.SQLCOLUMNS. Below is a data flow to copy SQLCOLUMNS to the ODS. Qry_Set_Desc just picks out the metadata columns we are interested in.

Extracting Metadata 1

Then I use a second data flow to limit to just the tables I am interested in and output the results into a format that makes them easy to transport to our target database. In this case I have created a 4 column file, although you could get by with just the last one if your prefer.

To download PDF and Continue Reading…

ernie-phelpsAbout Ernie Phelps
Ernie has 15+ years of experience in EIM with client in many verticals, including aviation, construction, finance, and healthcare. He is also a certified SAP instructor. At Decision First Technologies, Ernie specialized in SAP Data Services

Categories: Data Services