It depends on what kind of work you are looking for in a start-up? I have some friends who have started tech-start ups and from what I've seen they don't do much financial analysis. In a start-up generally you'll be working very broadly in financial management doing many tasks from accounting to collecting bills and even marketing if necessary. But of course everything depends on the company.
In early-phase companies, where there is not necessarily VC financing yet, the budget is generally strict which makes hiring of new employees hard. After the first VC round the VCs want to have their say in the hiring decisions and tend to look for experience and demonstratable talent.
If you're able to get into a company early on and help them raise VC financing it could be very interesting experience but the work generally entails long days working with many different areas and very little pay in the beginning.
Since start-ups have little resources, every recruitment is a big risk. However if you're thinking about PE/VC as a future path, it might be interesting to see the business from inside.
Many new business centers and incubators provide advisory services to start-ups. You could call the administrative staff for hints for where to apply if you want to cut down number of calls. Many start-ups have quite a few short comings and problems in the general business logic, so I would recommend to research the company well before you start working, if you don't want to end up spending your time arguing against a managing team of "tunnel visioned" engineers. (nothing against engineers, some tech start-ups just tend to be too much about the tech ) Many countries also have e.g. software entrepreneur associations and you can look through the member lists for potential companies.
Start-ups don't generally do very active hiring and post ads, so you should make the first step in contacting them. It would be good if you can demonstrate wide knowledge of managing the financial side of a company since start-ups don't have money to hire specialists for each specific area. Also VC contacts and previous experience in raising finance is looked upon very favorably.