If your business uses data from various business software, systems, and applications to complete vital parts of your operation, it is a must that your systems work synchronously—thus, a need for system integration. This strategy not only gets you ahead of the game, but it can save you time and money, as systems that are not integrated—and even those that are manually synchronized—are costly, requiring big resource and time consumption.
This blog will answer the following questions:
- What is system integration?
- What need does it address?
- How is integration done?
- What are your options when integrating your business software?
- How do you avoid spending too much money on integrations?
- Data – usually consists of text and numbers, but can also be pictures, videos, files, etc.
- Source system – the system that sends data to another system.
- Target system – the system that receives data from another system.
Why is integration done?
In general context, system integration is the process of transferring data from one system to another, often through the use of a third-party application.
Integration is done by programmers to automate the exchange of data between two or more IT systems, for instance, transferring updated prices or stock files from an enterprise resource planning (ERP) system to a webshop, or vice versa.
Why should integration be automated?
- To prevent people from making errors. There’s a high probability of creating errors when you use the copy-paste function.
- To save time. Copying and pasting data—especially for complex systems—could take up a lot of time. Likewise, to avoid importing large Excel files, which also takes a lot of work and time.
- To exchange data in real time, or at the exact moment that one or another data source is being updated.
Which way is data sent?
One-way or two-way integration can be done. One-way integration involves an application transferring data to another system, while two-way integration involves two or more systems transferring data with one other.
Who pays for integration?
The company that needs data integration shoulders the fees for this development.
Here’s a tip: if your project is about to exchange data with another company, aim to lead the integration process. With the integration under your control, you will achieve the data transfer you need easier, rather than just following the flow from the other side.
How are data sent between systems?
You can send data back and forth in many ways. These are the four most commonly used methods:
- Modules/plugins installed in a CMS, webshop, CRM, etc. This method is preferable among all other methods, though you may not get the 100% perfect integration. The reason is that integration works the moment you install the module. A rule of thumb: always use ready-made integration modules whenever you can. Consider arranging your requirements rather than implementing custom integration.
- Web services (API). Web services usually provide stable integration compared to export and import of files; but unlike using module/plug-in solution, web services require that you have an in-house or freelance programmer to develop the integration.
- Export and import files (e.g., XML or CSV files). Here, the source system composes a file stored in a folder on the server from where the target system can load the file. (It may be on the same server or over the internet.) There are different modules/plug-ins that fully or partially automate the integration process, but normally it will require individuals with solid technical background such as programmers to make it work properly. It is important to keep these in mind:
- If you exchange data files, remember to password-protect the folder where the files are saved. Ask your programmer to use IP blocking to reduce if not eliminate the possibility of others having full access to the files.
- Remember to encrypt a username and password if, for instance, the files are exchanged via FTP― a common way of exchanging data
- Saving data directly in an external database system. Here, the source system stores directly into the target system’s database. I cannot recommend this method even though it is often the easiest and fastest way to solve an integration project, as this method opens the risk of having security issues and programming errors, which usually occur when the target database changes. It allows the developer to make small changes in the target database that may result to fatal consequences.
How often is data sent back and forth?
If you use a module/plugin or a web service method described above, then data is sent as soon as the source system is updated. The same applies if you write directly to the database on the target system (however, as mentioned above, this method is not recommended).
In sending files like XML or CSV files back and forth, you must configure how often it should happen. For instance, once per hour or once a day. The target system can monitor the folder where the file is saved and import the data automatically.
Here are pieces of advice on exporting and importing data:
- If you export a huge amount of data (e.g., 100 GB) in one go, you may only be able to do it once in a day because both exporting and importing take time. But you can save time by exporting only the data that has been modified since the last export. Remember to remind your programmers about this.
- From time to time, it may be convenient to have a flash update from one system to another. For instance, you have updated the prices or stocks status of your products―having flash updates can be a good option for this. Consider asking your programmer to implement such feature.
What are the things you must ask your programmer?
- Which method of integration will be conducted? It can be through modules, web services, or file exporting―but do NOT let him write directly to the target system's database.
- Has the developer searched for a plug-in/module? Keep in mind that your developer may ask for higher fees if the integration is custom-made or programmed from scratch, as compared to him using a complete plug-in/module.
- Has the developer done data mapping? Data mapping is the act of plotting the fields and deciding which data is stored there. Many kinds of data are easy to map, such as a zip code.
Here’s an example of data mapping a person’s name: Suppose that the source system is quite an old system and it saves a person’s full name in one field, i.e., first name, middle name, and last name. Let’s assume that the target system saves names in three fields: first name, middle name, and surname; the fields could be called: [first_name], [middle_name], and [last_name].
When you integrate the old system with the new system, you must decide how to handle the names. Do you hide the name in a field, or do you divide the names? You have a few options:
- Data mapping comprising name of one word, so all is saved in the [first_name] field.
- Data mapping comprising name of two words, so the first word is saved in the [first_name] and the second word in the [last_name] field.
- Data mapping comprising name of three words, so the first word is saved in the [first_name] field, the second word in the [middle_name] field, and the third word in the [last_name] field.
One last, important note: always use SSL encryption for data exchange so unintended recipients can’t read it.
Integrating data from the various software, systems, and applications that you use in your operation is a smart business and technology decision that will hugely work to your advantage. It will cut down cost and hours for resources, increase efficiency, and reduce risk of errors, all of which give you edge in the competition.
1902 Software has been doing system integrations for years and has helped many individuals and businesses with this need.
If you have more questions about system integration or if you require this service, feel free to contact me by filling out this form.