Useful strategy documents - part 2.
So, continue the previous
post with the interface strategy document.
What are the useally questions, desires, sentences at the end of a projects if you haven't create an interface strategy document at an early phase of the project? Some example:
- "What type of new data source?"
- "You have never said that these data should be transfer to there"
- "It is impossible to deliver these data type to there, we haven't developed interface client for that type of data store"
- "Good, so we have finished testing the y data type, let's go to test xz data type. What do you mean that is impossible? It's wasn't developed? But I have tell you already 2 months ago at a lunch about this data type."
Et cetera, et cetera. I know you will have other good examples from you previous projects, just think back a little.
Interface strategy
The interface strategy document is quite similar as a migration strategy document, of course it has to concentrate on interface not on migration :) But it's structure, it's main purposes is very-very similar. Think a bit on this question: What a difference a data migration process and a one way once executed data interface process? The answer is usually nothing or not so many. So you will find below many-many sentences that I have copied from my prevous post with a small modification.
The interface strategy document is intended to provide a road map for performing the interface development between of old legacy systems and the new system(s). It should define the scope and objectives, the critical success factors and risks associated with the data transportation, transformation and data loads to support the solution. Of course it should describe the data conversion and interface requirements of the business too.
This document should be the first product of the interface team of a project. Never let them to wait until the main common requirements collection or system design document creation has finished, it should make parallely with this documents.
Who are the targets of this document?
- The project leaders who should accept the strategy and build it's results into the project 's plan. The managers use this document to understand how the interface team plans to perform the interface development, and how the creation of interfaces may impact the overall project. So they could determine what other project groups are concerned.
- The members of the interface team.
What is the mentioned minimal contents of a interface strategy documents?
- Logical diagram of the whole interface architecture and processes
- Data stores
- source systems
- target systems
- intermediate, temporary stores (for example intermediate data conversion, special stage areas)
- Lists of common data types per data modules as
- General ledges
- Inventory
- HR
- etc.
- Strategy plans for
- Data extraction
- Data loads
- Processing of transported data
- Data transportation method
- Data conversion - if needable
- Error and exception handling
- Testing
- Quality check
- Plans for the technical solution
- Data transportation technical tool(s)
- Custom developed and/or
- 3rd party data integration tools, frameworks
- Technical suggestions for data moving, for example
- Flat files
- Database tables
- Planned project resources
- Interface team origanization
- Software and hardware requirements
- Project standards
- Risks
- Interface acceptance criterias
- Exact requirements for acceptance
- Metrics for acceptance
- Minimal values for the metrics
Let's into the details
Logical diagrams of the whole interface, integration architecture and processes
There should be a diagram that show the whole final integration architecture, containing all participants: data stores, integration programs, frameworks and any other stakeholders
The same or another should show all interface transportation processes from a high level viewpoint. I mean it should represent how will the live interface system will transfer data how will it work after the going live date.
Of course allways put a short description after the diagram, to represent all objects, steps that the diagram contain.
This part of the document should be made first and finalize last. The first version of the diagram should give a very high level plan for the interface team to begin their work and the finalized version should summarize all changes what has made in plans and all information that has collected in other parts of this document.
Data stores
Commonly an invidual interface process has got source and a target data stores but sometimes there are intermediate data stores too. What should you put about them into a strategy level document? Of course not the technical details, so what?
First collect all datastores. Sources and targets too. How many times did you find in such a situation when the project's interface teams think that they are ready with the interface development, all data interface process has been created, developed, tested and checked in the new intergration architecture and somebody at the customer side suddenly ask a surprising question: "Ok that's right, but what about the ... system's data?". If you fulfil well this part of the document you could minimize the possible occurence this kind of situtations. Of course it could not guarantee that it would not happened, but how different situation if you could say "Ok I understand that it's required, but it wasn't specify/enumerate in the interface strategy document what you have accepted earlier."
As at migration never forget to enumerate all target datastores when you create the interface strategy document. It is still true here what I have wrote there:
Usually the solution provider's expert are extremly sure that they know well the target system, but are you sure? It isn't a wasted time if you spend some hours on collection of them. I promise you will head pin with you hand at least once "How good that I have not forget now, what could be wrong if I think on it only near end of the migration process"
Last but not least datastore are the intermediate data stores. These ones what you usually couldn't identify at first version of the diagram, these one comes out when the interface team made decisions on data extractions, data transformations and data load procedures.
Common data types
When you have collect the source and target data sources it is time to identify the to be interfaced common data types, groups in these data stores. Just put them into a list with a sort description. This table will very useful at design phase, for example every row could give the base of an interface xml file.
Strategy plans
As you have seen above strategy plans includes plans for
- Data extraction
- Data load
- Processing of transported data
- Data transportation method
- Data conversion - if needable
- Data quality check
- Error and exception handling
- Testing
Why they are important? I hope most of them speaks for themself.
Commonly every plan should include who, how and when will do the actual step. The strategy plans chapter should declare what kind of steps are factually necessary. Usally only the conversion step and the processing of transported data step is a choiceable step but it allways depends on the actual project goals.
Data conversion step means for me all data modification, converting and cleansing requirement. For example when target data store could accept the data if two extracted data concatenated, or on the extracted data has to run special algoritmic function to calculate the imporable data and so.
Don't forget to give enough emphasis on data quality checking. I don't mean to check for syntactic error I mean to check that the imported data are enough good for the target database lookups. Is it contain the right lookup codes, is use the common data dictionary, or not. Etc.
By the way never forget to think on that the new integration architecture need a common data dictionary or not, should it have an extra task for synchronising the data dictionary between the systems or not. Which master data where will be stored, which sytem are the source of a master data.
Allways put some thoughts about testing of interfaces into the document. Will the interface test made by hand? Could the interface team use any test tool for automating the interface testing? Will all interface development finished at the same time or the source and target interface code will be finished at different time so between these to date the interface team should simulate one of the interface to test the other part of an interface process.
Don't forget! Don't go into low level details. This document should make as fast as can, you will have time at migration design phase to extract the details, so just put here only the common, has to be decided, approved high level plan description.
Plans for the technical solution
It's time to go into some technical details, of course severely staying in high level too.
What you have to decide? These examples are very similar as at migration and it isn't an incidental. Think on it, migrating data is a special type of an interface process, it could use the same architecture as the integration, the only difference that it will runned only once (on most cases :) )
- What kind of technics do you want to use for extraction, for loading data.
- How do you want to implement the data conversion. Will you store the extracted data in an intermediate database, file. Will you have to develop a conversion tool?
- Is human resources needable for running an interface process or we could use algorithms for all integration steps.
- Do have the customer any data integration tool (Oracle Data Integrator, Talend etc.)
- Who will maintain the live integration architecture.
- How would technically documented the interface processes (for example where and how will the log data generated)
Planned project resources
The strategy documents has to contain information for what and how many resources want the interface team to use during the interface development.
Some of them already accepted in the project's foundation documents but it's good to repeat them here. So what I think is necessary here? These ones:
- Interface/integration team origanization. If you don't do it earlier then name all member of the team. (customer and supplier side too). Gave the exact details responsibility of each member of this team. Who will be the leader, who has to make decisions, who will be participate in the design, in the development, in testing, in quality check, in live migration.
- Software and hardware requirements. As much as possible at this level specify what kind of softwares and hardwares want to use the team during until going live date and after it.
- Project standards. Specify what kind of standard has to keep the team during the design and development, such as version control, code standards, naming convetions etc.
Risks
It's what is means. Try to collect any risks that could influence the planned interface process., integration architecture. Anything that could break it, slow it and so on.
Integration acceptance criterias
This is the hardest to part of the document. Usually the customer does not know how will they accept that the interface processes running well, the integration architecture is acceptable for running the live processes. They have got many requirements, many idea about the integration and usually they want to answer for questions that directed on acceptance. Try to not stop here, collect as much information as you could!
Some food for thought sentence
- Try to collect the exact requirements for acceptance of the integration.
- Try to indentify the metrics for acceptance. Which one is harder, a real acceptance metric, which one is just a to be measured metric.
- Try to set minimal values for the metrics, what values is acceptable by the client - which means what has to reach during the live migration.
Summary
I hope I could well summarized what does a interface strategy means for me. I hope too that it will helpful for you and you could use them at your project too.
At next post I will talk about the testing strategy document.
