Homeschool through primary grades, or no?

This debate is going to go through our house for at least the next 6 months, if not until next August.  Carolyn is flourishing and loving doing her preschool work at home.  She does basic addition, is beginning to read, can write almost every letter, can sort well, loves listening to history/social science books, and is enjoying science books and activities.  I read over 10 books to her every day, including 2 upon waking, 1 at nap time, and Jon reads 1 at bed time.  I also read out of a chapter book to her each night.  She loves every book, the current one being Matilda.  We've also read Charlie and the Chocolate Factory, Winnie the Pooh, The BFG, Mouse and the Motorcylce, and countless princess and fairy chapter books.  We just recently started on more of the classics.  I'm looking forward to starting Little House books after Matilda.  My point, I guess is, she's chugging right along, and she's only 4.  I might be wrong, but I think she's already beyond her age in terms of school.  She loves her art projects too - cutting, gluing, coloring, painting, sculpting with playdoh.  She interacts well with everyone she meets, including holding a conversation with the urgent care check-in lady while I paid our doctor bill last week.  So where do we go from here?

I seem to seriously think about homeschooling once a month or so.  But I'm always met with the same response from Jon.  "What's the rush" when I say that in homeschooling she'd learn things faster because she would be interested in the material.  No rush, but why make a child struggle to learn something because s/he isn't ready even though that's the grade that some arbitrary committee decided it should be taught in.  "What about socializing" is often asked.  This is one I get hung up on a little, because our current neighborhood doesn't have a lot of kids in it, at least not ones Carolyn and James' ages that are outside.  They don't have any real friends at this point, although Carolyn could put a list together for you if you asked her to.  I hope that they'll make friends through their sports activities and wherever we move I hope they'll have more friends on the block.  I do know that most kids learn bad behaviors in school, not things I care for my kids to start doing.  Do kids really need to learn to sit and listen for hours on end?  Learn to fit into the program and that their needs/wants don't matter?  Some might say that's part of life, but why does a 5, 6, or 7 year old need to learn that?  As they get older they'll naturally slow down.  Why stop a child from spending time upside down (Carolyn often hangs out in a handstand on the couch)?  From moving around while looking at a book (you should the positions!)

Deep down I think that I really want to homeschool, but I don't have the responses needed for my husband at this time.  I don't know if I ever will, because we seem to just be at opposite ends on this.  Why change what's not broken? is often said.  But in my opinion, there's a lot broken about our schools across this country.  I'm a product of some great schools, but I lost my passion for learning along the way.  I started to give up when things got tough to learn.  And I think that was because I was never challenged until college, so I didn't really even know how to respond.  When I tell people I majored in physics with a minor in math I typically get "wow" or something similar.  I love physics, but I wonder how much I could learn if I had learned the proper study skills/how to stick with someone at a young age.  I had my Master's by 22, but might it have been in science rather than education?  I'll obviously never know the answer to this question because I can't turn back time.  But I'd like to think I can help my children make the most of themselves, and this might be the way to do it.