Samson Q2U for about $60
Definitely an “it depends” question, but assuming the computer you already own is good enough to start a podcast with (8GB RAM, 2.5Ghz Quad-Core Processor should be fine), you can get a good podcast mic like the Samson Q2U for about $60, use free podcast recording and editing software like Audacity, and get podcast hosting free for 6 months from Fusebox — bringing the total to about $60 to get started podcasting!