Overall there isn't a sharp difference between the severity of viral infections coming from animals (zoonotic) or human transmitted because it relies on many factors such as the virus characteristics, pathogenesis, and the immune status of the individual.
Viruses that are transmitted among humans can be more severe because the virus would be well adapted to the human immune system and has already acquired the ability to use the cell machinery to replicate and grow.
On the other hand, zoonotic viruses can be hard to contain and pose great danger as well because they would be completely new to the immune system and would cause great shock among populations if the transmission rate increased (such as COVID-19 and how it caused a pandemic). It's important to mention that they're also harder to contain since the hosts are animals which are not at our full disposal to maintain.
So, as you see, both are dangerous in opposite ways. In the human transmitted case the virus would be more "experienced" to the human immunity, while in zoonotic diseases the immune system is very "inexperienced" to the virus.
I agree with Aia Sinno. Viruses tend to be stubborn, no matter the source. As mentioned in previous answers, several factors play a role (viral and host). If the host is disadvantaged especially in zoonotic viruses due to lack of prior experience with the virus (no immunity) and added on the pathogenesis (organs or systems affected), the viral infection might be severe. Look at the hemorrhagic viruses, they seem to be more severe and majority are zoonotic.
Viral diseases are mediated by cellular immunity. Disease is a function of infectious challenge versus host immunity. Anything that impairs cellular immunity enhances the probability of infection progressing to disease 9genetic composition, dietary imbalances, stress.