Tiny black caterpillars

When I sit outside, tiny black caterpillars, about 3/8" or 5mm long crawl up me. I don't recall experiencing this before.

Would this be the start of an infestation? And what natural thing could I do to get rid of them and to protect our fruit trees and our garden?