The costs of simply forming the business are low. For example, setting up as a sole trader can be done without any costs just by informing HM Revenue & Customs. If you want to form a private limited company (or a limited liability partnership) you can choose to do this yourself on payment of a small fee to Companies House, or ask your solicitor or accountant to do this on your behalf.
In a wider sense, of course, starting a business can involve wide-ranging costs. Initial costs may include premises, equipment, furniture and fittings, vehicles, insurances, raw materials or stock, professional advice and so on. You are likely to need to pay overhead costs such as salaries and utility bills, while it may take some time to build up sales (and receive payment for them).
If you need to raise money to help cover the costs, you should take advice on the best options and implications. In particular, you should take advice if you are asked to give a personal guarantee for any loan (or other contractual agreement).