You may refer to [ERDAS IMAGINE] software for identifying the pavement distress using close range photogrammetry technique, you can also refer to my published papers and book on this issue.
I think that we would need some typical images of distresses to employ into this learning technique to facilitate recognition of defects and distresses.
The problem is rather complex, there are some key pavement surface defects. The methods employed should rely on distinct features for each distress at the different levels of severity.
Pavement videos shall be collected by fixing a camera in a vehicle. These shall be converted into image frames and the distresses can be identified with the help of a MATLAB script.