Zoning and construction regulation has obviously not kept pace with the changing social expectations of housing nor innovation in building construction. And you are right, a lot of it was enacted (especially minimum square footage zoning) in response to trying to keep "undesirables" out of areas in the aftermath of white flight from urban centers in the 50s / 60s and has remained ever since, since the children of the generation that enacted those laws grew up being preached to how bad non-conformance is and they are now the largest voting block in America.