How can I relieve pain of multiple yellow fly/ horse fly bites?

I would put some topical benadryl or anti-itch to stop the itching. After that, perhaps an Ibuprofen to help with the swelling. Then I would call the local clinic and ask them.