From my experience, it depends hugely on the use case. You can have projects with the goal to increase revenue (eg. by up-sell, cross-sell) or save costs (eg. predictive maintenance). It will be done differently for different industries or business units like marketing and sales, supply chain, risk, finance, HR or any other.

Ideally, you should specify the project goal and metrics before you start implementing machine learning algorithms. Selecting the measure requires business and algorithms understanding. They have to reflect your goal and you should be able to translate them into ROI.

I'm not sure if there is a one size fits all approach.

Tomasz Smolarczyk thanks for the comments. Following your idea, we have to contextualize the business goals and align the Machine Learning projects with their contributions to those goals. The business objectives should then be the foundations or the reason for developing a machine learning project, which should provide the metrics to measure what to be improved.

Is there any report or study addressing the investment in machine learning projects?

Hi.,

To determine the return as a hard number, determine the profit from the project and subtract the costs. The final figure is the money made from the project. To determine the percentage return, divide the hard number from the latter calculation by the original cost.

For more info:

https://blog.usejournal.com/machine-learning-roi-how-to-model-the-investment-value-of-machine-learning-a29be1f2f827

Great C K Gomathy it there any paper or report addressing this subject? Manly related to Machine Learning projects ROI or investment being developed in this area by the organizations?

Thanks for the link!

No one will pay anything, because the developers of artificial intelligence systems do not work for money, but for their own pleasure. They can even be charged money to let them work, and they will still work even in these conditions. Summary: they will pay not for IT-projects, but for the effect of their implementation.

Eugene Veniaminovich Lutsenko thanks for your comments and agreeing to that the projects per-si are not the main idea but their effects (their contribution). To develop projects and justify to managers there is a need to explore the financial outcomes of the project, e.g. when netflix created a competition with 1 million dollar prize for algorithm to increase the accuracy of the company's recommendation engine by 10 percent, they expect outcomes to develop this. Considering that a project should be developed to contribute to a business objective (at least is developed with that in mind), the machine learning ROI could be measured according to its contribution to a business objective, e.g. reduce dropout in 5% and this could be considered the effect to achieve the objective. Is there any paper addressing how to evaluate or estimate the ROI in machine learning projects and investments? Or the investments being made by the organizations in this area?

There are several articles, and even books. In particular:

Book THE WAY I AND STYLE FEATURES OF ACTIVITY OF EMPLOYEES OF INT...

Book AUTOMATED SYSTEM-COGNITIVE ANALYSIS AND THE SYSTEM OF "EIDOS...

1. Lutsenko E. V., Korzhakov V. E., Ermolenko V. V. Intelligent systems in controlling and management of medium and small firms: Pod nauch. edited by doctor of Economics, Professor E. V. Lutsenko. Monograph (scientific edition). - Maikop: ASU. 2011. - 392 p. http://elibrary.ru/item.asp?id=21683734

2. Lutsenko E. V. Theoretical foundations, technology and tools of automated system-cognitive analysis and the possibility of its application for a comparable assessment of the effectiveness of universities / E. V. Lutsenko, V. E. Korzhakov / / Politematic network electronic scientific journal of the Kuban state agrarian University (Scientific journal of Kubgau) [Electronic resource]. - Krasnodar: Kubgau, 2013. - No. 04(088). P. 340 – 359. - IDA [article ID]: 0881304022. - Access mode: http://ej.kubagro.ru/2013/04/pdf/22.pdf, 1.25 cu.

3. Lutsenko E. V., the Implementation of psychological, educational and vocational tests and supertests without programming intelligent system "Eidos-X++" (On an example of the test: "analysis of the peculiarities of individual style of pedagogical activity") / E. V. Lutsenko, V. E. Korzhakov // Polythematic network electronic scientific journal of the Kuban state agrarian University (the Scientific magazine of Kubsau) [an Electronic resource]. - Krasnodar: Kubgau, 2013. - No. 04(088). S. 1057 – 1085. - IDA [article ID]: 0881304076. - Access mode: http://ej.kubagro.ru/2013/04/pdf/76.pdf, 1,812 cu.

4. Lutsenko E. V. Automated system-cognitive analysis as a method of complex solution of the problem of personnel management with the use of functional-cost analysis / E. V. Lutsenko, V. E. Korzhakov / / Politematic network electronic scientific journal of Kuban state agrarian University (Scientific journal of Kubgau) [Electronic resource]. - Krasnodar: Kubgau, 2014. - No. 02(096). C. 1 – 16. - IDA [article ID]: 0961402001. - Access mode: http://ej.kubagro.ru/2014/02/pdf/01.pdf, 1 U. p. l.

5. Lutsenko E. V. management of personnel with application of functional-cost and system-cognitive analysis / E. V. Lutsenko, V. E. Korzhakov / / Politematic network electronic scientific journal of Kuban state agrarian University (scientific journal of Kubgau) [Electronic resource]. - Krasnodar: Kubgau, 2014. - No. 04(098). S. 1009 – 1041. - IDA [article ID]: 0981404075. - Access mode: http://ej.kubagro.ru/2014/04/pdf/75.pdf, 2,062 cu.

6. Lutsenko E. V. Methodological aspects of identification, presentation and use of knowledge in ASC-analysis and intellectual system "Eidos" / E. V. Lutsenko / / Politematic network electronic scientific journal of Kuban state agrarian University (Scientific journal of Kubgau) [Electronic resource]. - Krasnodar: Kubgau, 2011. - No. 06(070). Pp. 233-280. – The cipher of Informregistr: 0421100012\0197, IDA [article ID]: 0701106018. - Access mode: http://ej.kubagro.ru/2011/06/pdf/18.pdf, 3 u. p. l.

7. Lutsenko E. V. Metrization of measuring scales of different types and joint comparable quantitative processing of heterogeneous factors in the system-cognitive analysis and the system "Eidos" / E. V. Lutsenko / / Politematic network electronic scientific journal of the Kuban state agrarian University (Scientific journal of Kubgau) [Electronic resource]. - Krasnodar: Kubgau, 2013. - No. 08(092). S. 859 – 883. - IDA [article ID]: 0921308058. - Access mode: http://ej.kubagro.ru/2013/08/pdf/58.pdf, 1,562 cu.

8. See also "Eidos" system mode: "4.1.6. The assignment of objects into classes (task assignment). Functional and cost analysis in personnel management. Set resource limits by class. Enter object costs. Assigning objects to classes (LC algorithm and RND algorithm). Comparison of efficiency of LC and RND algorithms".

Article AUTOMATION OF FUNCTIONAL-COST ANALYSIS AND THE METHOD OF "DI...

Eugene Veniaminovich Lutsenko Great :-) I will analyse that. Thanks.

The Eidos-X++ system differs from other artificial intelligence systems in the following parameters:

- was developed in a universal setting, independent of the subject area. Therefore, it is universal and can be applied in many subject areas (http://lc.kubagro.ru/aidos/index.htm);

- is in full open free access (http://lc.kubagro.ru/aidos/_Aidos-X.htm), and with the relevant source texts (http://lc.kubagro.ru/__AIDOS-X.txt);

- is one of the first domestic systems of artificial intelligence of the personal level, i.e. it does not take special training in the field of technologies of artificial intelligence from the user (there is an act of introduction of system "Eidos" of 1987) (http://lc.kubagro.ru/aidos/aidos02/PR-4.htm);

- provides stable identification in a comparable form of strengh and direction of cause-effect relationships in incomplete noisy interdependent (nonlinear) data of very large dimension of numerical and non-numerical nature, measured in different types of scales (nominal, ordinal and numerical) and in different units of measurement (i.e. does not impose strict requirements to the data that can not be performed, and processes the data that is);

- contains a large number of local (supplied with the installation) and cloud educational and scientific applications (currently 31 and 159, respectively) (http://lc.kubagro.ru/aidos/Presentation_Aidos-online.pdf);

- provides multilingual interface support in 44 languages. Language databases are included in the installation and can be replenished automatically;

- supports on-line environment of knowledge accumulation and is widely used all over the world (http://aidos.byethost5.com/map5.php);

- the most time-consuming computationally, the operations of the synthesis models and implements recognition by using graphic processing unit (GPU) that some tasks can only support up to the solution of these tasks is several thousand times that really provides intelligent processing of big data, big information and big knowledge;

- provides transformation of the initial empirical data into information, and its knowledge and solution using this knowledge of classification problems, decision support and research of the subject area by studying its system-cognitive model, generating a very large number of tabular and graphical output forms (development of cognitive graphics), many of which have no analogues in other systems (examples of forms can be found in: http://lc.kubagro.ru/aidos/aidos18_LLS/aidos18_LLS.pdf);

- well imitates the human style of thinking: gives the results of the analysis, understandable to experts on the basis of their experience, intuition and professional competence.

In addition to ROI, there are other private criteria of knowledge and they are used in the Eidos system in the development of system-cognitive models. About this there is in article (and in other):Article METRIZATION OF MEASURING SCALES OF DIFFERENT TYPES AND JOINT...

Thanks for the links. Is there any translation or articles in English? From the links provided, I can see the abstract in some of them in English. The idea is to develop some theoretical background addressing the investment that the companies do in machine learning projects and criteria selecting the project according to their expected return of investment.