He isn't really that big if you look at him in relation to leia and han. Worst case scenario is scale him down to about the size of zero the hutt in the clone wars which doesn't look too bad.
He would look too bad, you can trust me on this one. For one issue you fix, two new ones would emerge.