Hi all,
At the beginning of the ETL design project, very often organizations are required to decide about the architecture of the ETL system and this includes Africa, looking at the sensitive of the dataset that the DW of Africans MoH deals with, what would be the preferred choice for MoH, custom design or off the commercial offerings?
As the data for African MoH is unique in nature, one would be inclined to look for specific healthcare ETL solutions in the market but as we all know, these systems normally come with huge annual license costs, and other disadvantages which may impact delivery of MoH DW. What are your view on this issue?
Best,
Mohammed.
Hi,
Everywhere cost is one of a determinant factor in data warehouse ETL process.
I am imagining Grid and Cloud computing environments as it has been the recent focus of attention of a large part of the scientific community, which was interested to achieve better performance and reliability in everyday business operations through the use of low-cost computational power to data-intensive operations according to Nouha and Abdella(2015). Resource demanding operations, such as those present in the fields of health and astronomy, can benefit from this approach since they deal with large data sets requiring enormous computational resources to process them
Determine the cost of cleaning the data before cleansing all the dirty data, it is important for us to determine the cleansing cost for every dirty data element.
According to Guru99 (2018), to speed up query processing, I will have auxiliary views and indexes. To reduce storage costs, I will store summarized data into aws redshift. Also, the trade-off between the volume of data to be stored and its detailed usage is required. Trade-off at the level of granularity of data to decrease the storage costs.
In the end, I want to say that this is something I continue figuring out and please if you have any idea on it, would like to hear about it more.
Best,
Mohammed.
References:
Guru99 (2018) ETL (Extract, Transform, and Load) Process [Online] Available at: https://www.guru99.com/etl-extract-load-process.html#9 [Acceessed 24 March 2019]
Nouha Z. and Abdellah, Z. (2015) An Exploratory Investigation of Factors Influencing Data Quality in Data Warehouse[Online] Available at: https://ieeexplore-ieee-org.liverpool.idm.oclc.org/stamp/stamp.jsp?tp=&arnumber=7483222 [Accessed 25 March 2019]
- Badges
- Science topic