Q&A: Pershing's Patrick Yip details successful BI expansion

He offers lessons learned about scaling the system to 78,000 users

Pershing LLC, a company that provides securities clearing, execution and settlement services to institutional investors and investment advisors, has scaled its business intelligence (BI) system to 78,000 users over the past few years -- and is now planning to further expand its focus on BI. Patrick Yip, a director at the Jersey City, N.J.-based company, talked with Computerworld early this month about scaling the company's WebFocus BI tools from Information Builders Inc., measuring the performance of the system and the company's future plans. Pershing uses the BI tools to give investment advisors an aggregate view of account balances, holdings and trades that have been placed, eliminating paper-based reports that users once had to sort through.

Excerpts of the conversation with Yip follow:

You have scaled your BI tools over the past several years to 78,000 users. What have been the key factors in allowing you to accomplish that? You have to find someone who has done this before -- especially when you have a lot of users hitting your database with a lot of queries. You need to work with [lines of business] to convince them that you can't have everything. You can't sit in front of this reporting tool and say you want to query everything. You have to put some parameters around it so you can optimize it.

We treat this as a mission-critical application because if there is an outage, it automatically goes through a structured, managed process. It doesn't take long before the CIO and sometimes the CEO get on the call [regarding an outage].

With so many users accessing the system, how do you monitor it for usage and for performance? A lot of traditional BI applications have been treated as second-class citizens from the monitoring standpoint. We have our own tools we use to monitor end-to-end and use [of Information Builders'] Resource Analyzer on the back-end so we can know which databases and schema are being accessed. We have our in-house monitoring tool to make sure the loads to the data warehouse are completed on time. In real time, we do have a pretty formal process every morning to make sure the application is working and the analytic reports are coming up the way they should be. We have weekly and monthly scorecards, and we always are looking at the usage. We break it down by the top brokers and dealer firms that are using the system.

How are you expanding the way you use the BI tools? It is becoming more and more mainstream and embedded in business folks' day-to-day operations. We have applications that are taking data at near real time and doing transformation on the fly -- such as requesting a check or a federal funds wire. We have the BI applications that help managers identify throughout the day how many of those transactions are being requested, how many have been processed and how many are still pending or have been there for several hours.

What are some of the lessons learned around BI? It is more important for BI applications to get out there sooner rather than later. A lot of projects traditionally have taken a long time to finalize the requirements and get out there. Get BI impementations out there as quickly as possible. The 80/20 rule works pretty well. The day you put it into production, the requirements will change.

We have quarterly release for reports; make sure you measure what you are doing. We have made tremendous performance improvements on some of the reports ... to the tune of 500%.

Copyright © 2006 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon