I wouldn't beat yourself up too much about adding to the unemployed. If he only works for you one day a week, he must have other clients to support him...

That said, I feel your pain, we usually employ people through agencies specifically so we don't have to fire them ourselves, just ask agency for a replacement. Spineless, I know.