<script src="https://maps.googleapis.com/maps/api/js?key=xxxxx" async defer>

The script is taking time to execute and before the script is executed your html is rendered. In that case you need to wait till that view is not fully initialized, So you need to use the afterviewInit method.
afterViewInit is executed when component view is fully initialized so you will not face this issue again. And your code will be like this.

ngAfterViewInit()
    var map;
    map = new google.maps.Map(document.getElementById('map'), {
      center: { lat: -34.397, lng: 150.644 },
      mapTypeControl: false,
      zoom: 8
    });
}

Support On Demand!

Angular