You can read the book of chemical process industries, process calculations (Himmelblau). It will be useful. Its a vast topic, you need to know and understand the unit operation you are interested in, details of reactant, product and the time and temperature at which the reaction proceed forward or backward or may be irreversible, composition of the feed, composition of the product and reaction pattern that can say some details about the intermediate products (if any). Once you know everything you can make an energy balance and mass balance. Mass balance is super simple and straight forward, to calculate the energy balance, you need to know the chemistry involved and information on the system, like the free energy and how it change and the internal energy, so on. Its very difficult to answer here as you need previous knowledge on this topic and you have to read the above said text book.
You probably need to be more specific but basically it comes down to mass in = mass out (both total and each individual element) and enthalpy in = enthalpy out. Here is a link that might help: http://www.enggcyclopedia.com/2011/01/heat-mass-balance/
The only slightly tricky part about the heat balance is that you need to cool the reactants down to the reference temperature (usually 25 C) then add the heat of reaction and the heat recovered from cooling to the product to bring it up to the correct reactor outlet temperature. Sign convention is awkward for many students negative values are exothermic.
Mass balances are easier each element (C, H, S, N, O etc) must balance that is the exact same amount in and out in whatever units are convenient for you (lb, kg, kg-moles tonnes etc)
If you want to become a master in mass and heat transfer, my sincere suggestion is that, first you read and understand about basics in thermodynamics like ....system , surroundings, types of system, concept of equilibrium, and interactions between the system and surrounding.
Most college level chemistry books should contain the info regarding mass balances. General rule of thumb is to start with O then H.
As for energy balance I would suggest Introduction to Chemical Engineering Thermodynamics by Smith, Van Ness, and Abbott.
Perry's Chemical Engineering Handbook would also serve as a great reference. The 8th edition is probably the most user friendly but I would advise using the 7th or 6th editions for more in depth stuff.
Good economic evalution need good mass balance see Coulson and Richardson's Chemical Engineering Sixth Edition Volume 1 is also quite helpful for Heat and Mass Transfer as well as Fluid Flow this has flow sheet based , excel based limeralized simplified mass balance , instead detailed calculations , based upon split fraction
Several interesting 'case studies' dealing with the thermochemistry of chemical reactions have been presented for discussion at this forum, to which you may possibly find of some interest to look at (e.g.): https://www.researchgate.net/post/What_is_the_heat_reaction_for_Ammonia_gas_scrubbing_in_60_nitric_acid_solution https://www.researchgate.net/post/How_do_I_determine_what_the_degree_of_dissociation_is_for_hydrogen_molecule_in_561K_and_88_atm_in_water https://www.researchgate.net/post/how_can_we_calculate_gamma_specific_heat_ratio_for_real_gas_chemical_non-equili https://www.researchgate.net/post/How_would_you_calculate_the_heat_capacity_of_a_1_wt_KOH_and_13_Ligin_solution_I_have_the_cps_of_dry_KOH_and_Ligin_can_I_sum_up_by_relative_weight