You can get some reeeeeally nice results with an old film camera and 50mm lens, sure you'll have to spend money on film and developing them but it's not tooo bad. At least you'll get them printed whereas a lot of digital photography only sees flickr.com and you'd take the time to get a hella nice shot. Quality over quantity kinda thing. E.g. a Pentax k1000. Prolly (£/$)100 <-- Cheap lenses
Alternatively you have bridge cameras which if you can imagine are a regular compact but with a shed load of extra controls and much larger body, much bigger fixed lens. (£/$)150-300 <-- Least favourite option but still one to be said, likely to out grow camera quickly
Or, or, orrrr 2nd hand dslr but don't go beyond 4-5 years as that's when technology really shows it's age. (£/$)200-400 <-- Can buy lenses forever but £££
The film option will most definitely be the cheapest.
But! I'm not really sure what you're after so i dunno how useful this will be