Thinking of starting my own SEO freelance agency

Is this a realistic thing to do, i have almost 2 years experience and finding a job in the industry is a pain in the ass. Is getting my own local clients and starting a small agency for myself better this time around? Im mainly asking the people who have done, have alot of experiance or stories to tell about setbacks, and upsides.