After a successful opening spell things became more difficult and the club was eventually relegated back into the Port Talbot League in the late seventies.

A period of ten years passed when the club competed in the Port Talbot and District League.

During this time the club grew in stature on and off the field and in 1988 was re-elected back into the South Wales Amateur League.

In the 1990–91 season the club were promoted to the Welsh Football League and on 24 August 1991 played their first match against Milford United.

Since the club's elevation into the Welsh League, steady improvements have been made both on and off the pitch and in 2000 floodlights were erected.

Further history was made in the 2005–06 season when the club lifted the Welsh League First Division Championship, only losing three games.

In 2008-09 the same situation occurred with Goytre winning the Welsh Football League for a third time, but were still denied promotion.

Goytre got off to a bad start with the first game of the season which was in the Nathanial MG Cup taking a hefty defeat to Trefelin which ended 6–1 with Youngster Alex Walters scoring the only goal.

Karl got off to a great start by beating his old club Llantwit Major 4-3 who were sitting in the top half of the table.

Note: Flags indicate national team as defined under FIFA eligibility rules; some limited exceptions apply.