Text this: Managing the Pre-Training Phase