Work from home.
In fact I worked from my dark and dingy attic. An office is a 'nice to have' at this stage and an unnecessary expense. Do your best to try to avoid meeting in person all together, but if you have to meet you can do so at your client's office or hire a conference room. The costs won't come close to office rental.
Don't hire admin staff.
There is a temptation to hire admin staff to do all the crappy jobs that you don't want to do. This is probably a waste of money at this stage and any employees you take on should really add value. If you just need somebody to be available to answer the phone, then you should look at one of the many telephone answering services or virtual receptionists that exist. We initially used answer.co.uk in the first few months who charge on a 'per call' basis.
Don't buy software.
I'm not advocating piracy here, but for a new startup business much all of the every day tools you will need: e-mail, calendar, word processor etc. are available for free (Google Apps). Other tools such as your CRM will be available for a small fee per user per month, so for one or two people should be pretty cheap.
Don't miss any opportunities.
If you've managed to get a customer to call your phone number or visit your website then you better be there to deal with it. If an enquiry came in we dealt with it any time day or night. I once called back an enquiry at 9:30pm on a Tuesday evening and it turned out to be the CFO of a £500m property investment company. We won the work entirely on the back of calling back in the evening rather than waiting until the next business day.
Do what you have to, to survive.
Clearly I was never going to fund this startup venture with just a month's salary when I had a mortgage to pay and a family to support, so I needed to find some form of income. I had a good background in IT and found a contract that I could do 2 or 3 days a week. I had to travel and the work sucked, but without it we wouldn't have made it to the end of year 1.
Be mega-efficient and automate everything.
Get this right at the start and it will make things a lot easier (and cheaper) a little further down the line. We put the effort into making everything streamlined in the early days by automating invoice generation etc. When we got a surge of signups this proved invaluable.
Don't waste time.
At networking events for example. Lot's of things you may read may indicate that businesses are founded on relationships. I never went to any networking events and have always figured the time is better spent improving your product and service than pretending to like somebody in the hope they will buy from you. Time is your most valuable asset at this stage, so it's best not to waste any of it.
Don't waste money.
This seems an obvious one, and ultimately it is what most of these tips boil down to. But before you buy something you should really ask yourself if it is really necessary. Do you really need business cards? Do you really need an accountant? There is a temptation to get all the *things* you think a business should have. It's definitely worth just stopping and thinking about whether you have an actual need at this stage or not.