I can't give you specific advice for your location, but I can give you some idea of how things work in my County here in Georgia, if you want to be considered legal by the various authorities.
Disclaimer: I am not an attorney or accountant!
For the Tax Man: the Feds permit you to use SSN as a Tax ID if you are operating as a sole proprietorship. Your state income tax people may also allow this, but the sales tax people will assign you a sales tax ID #. To be legal, you will have to send appropriate sales tax payments monthly or quarterly. Even if you aren't charging your customers sales tax, you are required to pay it.
Typical cost: 5-7% of sales.
For the Licensing Dept: your County and/or possible City will probably require you to buy a business license.
Typical cost: about $100 to start
For the Bank: it's best to have a separate checking account for your catering business, especially if you operate under a name that is not your own (i.e. Super BBQ). Most banks won't cash a check that isn't payable to you personally.
Typical cost: a box of checks
For the Law: to be legal, you have to file with the government if you wish to operate a business under any name other than your own. If you want to be John Doe, caterer you don't need to file, but if you want to be Smokin' Doe's you need to file DBA (Doing Business As) documents.
Typical cost: $100 from LegalZoom.com or see your lawyer.
For the Health Department: in my state you are supposed to have a kitchen that passes the HD inspection in order to cater. Your home kitchen will not be acceptable; you must have a commercial kitchen, be it fixed or mobile. This means three bay sinks plus handwash, commercial grade appliances, etc. Very expensive to build yourself, however you don't have to do so. You can rent time in a commissary kitchen that has passed inspection.
Typical cost: varies by market. Start with http://www.culinaryincubator.com to find a shared kitchen near you.
For Yourself: you absolutely must protect yourself from losing everything should someone decide to sue you because they sat on your hot firebox or experiencing mental anguish from gaining weight after eating your pork. Visit an independent insurance agent who can sell you the special caterer's liability policy you will need. A million dollar policy is bare minimum these days.
Typical cost: about $200 per year.
With the exception of the liability insurance, it is possible to fulfill each requirement as you are forced to by officials or circumstance. Just be aware that if operate without being in compliance retroactive punitive action is possible. For example, catering small private parties for friends and neighbors may allow you to fly under the radar,
until one of the invited guests turns out to be a health inspector.
Again, the above does not constitute legal or financial advice - check with your CPA and lawyer before acting. Hope this helps!