I know that to calculate the yield strength I need to find the elastic region of the curve and it's slope and then use the 0.2% offset method. But what if my stress-strain curve is not exactly linear in the elastic region?
I would find the linear portion of the curve just above the first "kink" in the curve (y-value~20) and use the 0.2% offset rule. That should give you a conservative value for the yield strength.
This type of curve occurs on steel samples made of materials that were subjected to strong plastic deformation before testing, exceeding the elastic limit, but not reaching the strength limit. If you need a canonical curve, perform heat treatment with recrystallization - restore the original state of the material.
If you are worried about the slightly curved section, then in vain, it is never perfectly flat on real samples. This is especially noticeable on materials with an incomprehensible number of dislocations and zones of internal stress - to which yours can be attributed. I support the answer given by Mark Thomas Summers.