How can I find out what licenses, certificates, inspections and all that other stuff is required for my business?

One of the simplest and easiest ways to see what kind of licenses and other stuff is needed for your business in California is California Business Permits Made Simple. Of course, if your business is somewhere else, the SBA has a nifty link to find the right resource.

Now they don't cover every type of business, but they do cover a lot and by choosing a similar or related business you can get a list of permits that you need to consider for your business on a City, County, State and Federal perspective.