I have to agree with you. The curves just don't complement that many vertical and horizontal lines.
It seems like the background fabric is taking over your patchwork a bit... maybe trim the borders and add a checkerboard or some narrow piano keys to get the size you want?
I don't care for the curves either. the fabric is too overpowering for the small center. Maybe trim it back and use a solid (or reading solid) print to calm it down and focus back on the hst's.