Mostly for object tracking the measure accuracy is not used very common, another measure which is in consideration is the IoU, intersection over union as you are tracking it should be determined that how much region of your target is in intersection with the bounding box.
I am going to assume that you know the ground truth bounding boxes of the objects. In this case, you could use metrics such as mean average precision, intersection over union, or euclidean distance between the bounding box centers averaged over time.
I am not sure if you already looked at this github project, I found it to answer your question to some extent: https://github.com/rafaelpadilla/Object-Detection-Metrics