How to understand any Business Intelligence project

Everyday associates are facing this question. Only few lucky people start a project and continue to be a part of it till the end. Many of us join the project in between and some unlucky folks join in climax. Whatever the time you join, you always struggle to get a good understanding of the project.

                You may be a developer, may be a manager or may be a business analyst; I believe we all face similar kind of problem as we don’t understand a project very well. There might be a well-documented understanding pdf will be in-front of you but that hundreds of pages will not help you out, if you can’t ask good question to correct people.  

                Sometime people think that when the pressure comes then people learn fast. May be it is correct in many places but can you actually learn in pressure. In pressure you only gulp the details and try to memorize it. Remember memorizing is not learning. Learning comes observing the detail and absorb the extract from it. Learning is the only thing which can lead you to solution.

Then how to start?

Every project or every machine (In terms of Automation) runs on a simple logic Input-Process-Output. But first of all you have to understand the requirement – Functional requirement, which actually drives the whole force.

                IT or BI doesn’t run business. Only facilitate and help business decision makers by keeping and retrieving data in right time. On first day in a new project we should get these understanding that

• Why business is hiring us?
• What data we need to keep?
• Which are the business drivers for this project? 
• When we need to deliver?
• Who are the contact people for everything?

Now, you have the questions but you may get answer as follows:

•  To create a reporting system for an asset management system.
•  Their business data is from different sources.
• You will get a KPI document if you are lucky.
• There is a timeline and your project manager can deliver the plan.
•  Most fuzzy question! You may come to know slowly.

If you are a developer and joined in middle of a project or replacing someone, he can transfer some knowledge to you. He also can introduce with someone who can give you something. But trust me; no one can help you out if you don’t have any plan of your own to learn a project.

                The last question is about the contact person which is the most important in any type of professional work. You are not a God and you can’t learn everything. But you have to know that who knows what. In Business Analysis term it is called Elicitation- Reaching right people.

                You have to know the concerned person for every piece of knowledge you need to gather to work with. It is very easy to say that it is not my job. You can concentrate on your technical responsibility and taking a mock-up placed in your document repository you can create a report without thinking of the business manager’s expectation. Their expectation is already placed in a business requirement document.

                Yes, you can. But to add a value to your customer you are always welcome to propose something which comes from the knowledge over their demand. You have to understand the demand first and have to produce what exactly what they need and then the scope of improvisation come. 

                For that you need the person/people who is or are the expert on the subject over which the BI will be implemented. But whom you need and what you will ask to them? Before going to business people you need to do the following tasks.

• Tear down the project concept.
•  Do the reading and Dumpster diving
•  Detailing the needs.
• Prepare the questions.
• Find out the persons.
• Meet them personally.
• Post meeting documentation

Tear down the project concept

Divide and rule is the best process to get a whole grab in a project. But to divide it you need time and material. In different phase of a project you will get different kind of material available or prepared. So you cannot generalize this topic.

In beginning of project

In beginning of project you have nothing in your hand except the business needs which is also not clear as customers normally don’t know what they actually want. Their demand slowly increases when they start familiar with the new system which you are building.

                You need to start from arts not science. Tear down their demand. Let’s take an example of a reporting project which will track over sales of some different brand of cars.

                As a business intelligence person you should understand that though the project output is some reports which will show the sales from different region, but this is not the ultimate goal. They need this information to take the decision over dealership that are bestselling or worst selling and take action on them.

                Now you have idea that your output will be the input of some other system. So the numbers which you keep is the source of another financial system. The decision will be based on the turnover and that will be calculated somewhere else.

                To create the system you need to know the source and destiny of the data. You also have to know the business rules of the data to process in-between.
                Apart of that Business Intelligence needs an enormous infrastructure support. You don’t have to know the server mechanism or the details of the shared services but you need to know or build the architecture of the whole project.

                So for you here is my list of question, it grows anytime and number may grow exponentially:

What are the data sources and type	Data sources need to be documented as the source can be heterogeneous and geographically separated. It may be a yearly spreadsheet, access db in a user desktop or simple a regular changing text file.
What tools are going to use?	If it is not decided yet, decide it. If decided then understand the compatibility with other tools. Find out the challenges and limitations technically (like Business Objects XIR3 cannot create report from different sources from a single universe) This is important because you cannot change a tool middle of the project and you cannot learn a tool when it is not comes under your skill domain.
Where is the data center?	Data Center location is sometime important when you are doing a global application or serving a country data in 4 different time zones. A real life problem is in US, the year ending process completes before the actual year ends. Because New York and Los Angeles is having a time difference of 3 hours.
Top-Down or Bottom- up or Hybrid?	If the process is not yet defined then you need to define the process, which way to go. Kimball’s or Inmon’s or both.
Security architecture	Who to see what and what not see is need to be understood beforehand. The layered architecture with security levels need to be defined first.
Details of the infrastructure.	The biggest set of information if the infrastructure is ready. The data storage capacity. Required deep links. DEV, Stage, Test, Prod environments and their credential Database schema names and credentials External data feed process and source connectivity. These all you needs to know with additional information.
Document repository	Business Document repository Your company document repository. Technology document repository (E.g SAP SDN)
Client location and time zone	To communicate in proper time.

In Middle of a project

If you join in middle of the project the above information is already available but you need to find out the correct place to find these documents. You don’t have to bother about how to get the information but which place the information is kept.

                You also need to analyze the project situation by tear it down by the blue print of the BI structure. The methodologies and architecture followed by the project presents a flow diagram to you. You need to analyze your scope of work and gather all the information required to achieve your goal. As an example if you are a reporting person you need a database mapping document which map the database tables to the business objects. But some kind of general information everyone should gather.

Assuming that you join when requirement gathering and analysis is done and development is started here is my list which you need to find out:

Current situation of the project	In which phase the project is going on. What are the challenges the team is facing. What are the lessons learnt and best practice followed.
Project Document repository location	Lots of document created or gathered by Business Analyst and SMEs and put in some document repository. You need to know which folder have what. Download whatever you need and also download something you think you need but no one suggested seeing that. Sometime irrelevant information also important to understand a project.
SME locations and information	SMEs are most important dictionary for the business rules. Though everything is documented but they are the fastest way to solve a problem when functional understanding needs to meet technical challenges. In every projects quick adjustments is always needed to overcome technical challenges and put an interim solution which is aligned with business demands. Quick sheet which tells about contact information of SMEs solve the problem of real time searching for a correct person. That will save lots of mail chain and mail forwarding.
Team information and contact details	Sometime developers and managers are geographically separated and chatting and phone call is the only media to communicate. If you keep the contact details handy, you may skip many problems. This information is not only for BI. It’s for all the projects best practices.
Infrastructure	Take the architecture diagram. Check what you need from the whole setup to achieve your goal. Ask for the details to your technical colleagues.
Functional/Technical specification	Read thoroughly these document if prepared. Suggest changes if require.
Database mappings	For ETL developers you need a mapping document from data modeling to database tables. For reporting guys you need a mapping document from database to report element mapping. Without this information if you precede you need fur more time to give a solution over a simple workaround.

Do the reading and dumpster diving

Reading thoroughly and thoughtfully is mandatory skill in BI industry. BI is really not about the coding. BI projects are more aligned to business than technology. Understanding business demand requires a huge knowledge level of the industry which you are implementing the BI solution. Though you are a domain expert and have lots of certification in BI or several domain and business Area but every single project have separate set of knowledge which you have to gather, either by reading or reaching the business users.

               In onsite – offshore model, offshore don’t have access to business users. Very little time they have some interaction and also on some specific solution of a technology related problem. Onsite person also not get chance to ask all the questions which comes to their mind. Business users who actually use the system have their own goals to reach. They don’t have time to give you knowledge transition. They also sometime skip this responsibility.

                You have only choice to read. But what to read and what not to read they can help you. They can definitely show some location, some websites to get understanding of their terminology and demand. Also in period of time they create many documents which they used to upload in the shared document repository. They will show a location of one document, you need to grab the whole which is present there. Later you can delete whatever is not needed.

                Start with the document which you gathered with help of business users. After reading project related document and business document in parallel you will start understanding on your own that what you need and what you not. Then you can focus on the document which actually need. You can use MosCoW analysis for this though it is not what it is intended for. But once you classified the documents, you are half-a-way done.

                Once you get access to their shared document and grab the whole folder, you are actually doing dumpsters diving. People are always afraid to delete something. We used to keep everything though the document is old and new version is loaded. Use that weakness of human brain to your strength. Get the old document and check somewhere you may find something which gives you some light. As an example I was checking for a information that how all the car selling program is made. They should have some similarity. Because in any big organization idea evolves from idea, very few times a new idea popped up and implemented. Keeping that in mind I get all the documents about car selling program. Reading and classifying that information I came to know that a program is aligned to the brand market and technology. An electronic vehicle never part of a program of minivan selling. So you need to build the table like that which will accommodate all the new programs with different brand, markets and technology.

                Another piece of knowledge you will get from the customer desk. We will discuss it later. But as an IT guy you may not decorate your desk as your laptop is your desk. But the business people who work for a business more than 20 year in a same office put marks on his desk. So pick those and question to know them and their area of knowledge better.

Detail the needs.

 Once you are done with you reading (though it is impossible), start creating your understanding document. Understanding is yours, document will be yours. No prescribed format is present as of date. If you consider BRD as your understanding document, you should elaborate with such a detail which may help you to backtrack.

                To me documentation should be done in three ways

Excel documents	Data which you need to sort or classify. E.g: List of reports Number of users. Name of users by department. Test case links or test cases. Database mappings Most of the document should be in spreadsheet.
Word Document	Data which you need to explain in detail and write functional information. e.g. Detailing functionality Object description Overall details of process flow Business understanding
Visio document	Data which you need to visualize and memorize as summary e.g. Pictorial representation of all of your understanding. Architecture diagram Data or process flow.

Documenting your understanding in your way will help you to find broken links very fast. We used to forget things which we don’t use for some time. But if we read something regarding that will kick up the old information in our mind.

Prepare the questions and find out the person

Prepare your questions when you are reading. It is one of the proven practices of reading. Write your question in another paper when you studying a page. You may get your answer after reading the whole document but you will find some question which is not answered by that piece of documentation.

Before going to users read all the documents and listed down all unanswered questions. You should start categorizing the question by user. According to your understanding you know or assume the knowledge level and area of users. From your list of questions try to put the question by their name. After completing the categorization you will see some leftovers which may not fit with any user’s knowledge scape. Don’t try hard. Try to show the question to your user and ask that who can answer these. You will get the answer.

So the steps are:

      1.       Prepare the question dump as you read

      2.       Prepare the unanswered question dump.

      3.       Categorize questions by user

      4.       List uncategorized questions.

Find out the person and meet them physically

Though you listed down the questions and know whom to ask what, still you need to find those persons. And while doing that your list of people may change as people change their positions but the document never get updated.

                Once you are ready with the user list first verify with company directory about their role and existence. You may find someone who leave the company and you are in dark that whom to ask these questions. In organizational hierarchy you will find the managers of that person. Send mail and get the information that who replaced that position. You will get answer.

                Now you may find that your user may present in company but he switched the role few months ago. Then you need to ask them that who replaced them. You will get answer.

                Last problem you may be geographically separated with targeted user. Don’t bother about this. Use technology, use webex, lync, go-to-meeting, live meeting to connect with them. You need to check your system with your technology before asking for user to join. Once you are ready then sent them a meeting request with details of your questions, so they will be prepare with the answer before you go.

                Now the same thing will apply for visiting a user personally. But added advantage is face-to-face meeting is you can read his/her expression and shoot questions which are relevant and irrelevant to your project. Irrelevant questions is sometime more important than relevant questions as that will grow your knowledge of their demand. Let’s take an example, for a report requirement I came to know from a user that they need reports in pdf and excel. Pdf for their customer and excel for their internal users. While discussing other stuff about the day to day problem they are facing, he told me that they faces query on the sanity of the report data regularly. When they are sending reports to their clients they need to send the report in pdf format. But those pdf data is also getting tampered by pdf editors.

Post meeting documentation

You are almost done. You have plenty of notes and a fair understanding on almost everything you want. Now you know what they input, what to process and what they want as output. That’s where you start thinking on “HOW”. Till this point you were either a puppet or an order taker only, not an BI person. You have all your technical skill to answer the “HOWs”. But wait, did you documented what you just clear out from the SMEs. If not then do it now.

                You may think that the knowledge you gathered from the SMEs are the answer of your few listed question which you didn’t find from anywhere, and you will never forgot those. You are wrong. Write up a summary of the discussion in peace. You will find many things which are not answer of your question but can be useful for many problem solutions. Also, trust me you will forget if you don’t write as those questions are the toughest question which no previous documents explained to you. Brain is by default lazy and the best exercise is to let brain know that where you kept those answers.

I hope you climb up the learning curve and ready to put your analytics skill which doesn’t include the entity called People (which are the most difficult things to handle). If you follow the process you will come up with thousands of Jargon which really created for showcasing BI world. Just focuses on the layman terms described above and from the knowledge you gather create visual heavens from the chaotic data.

Good Luck.