When a supplier sends you an invoice, it’s up to you to ensure the ABN is correct and that they are registered for GST. 

You can check this by going to abr.business.gov.au or type ABN lookup in Google. If your supplier is not registered for GST and they charge you GST, you cannot claim the GST credit even if you pay the supplier in full. The supplier has no obligation to pay this GST to the ATO, in reality they have just charged you more for the goods or services.

Hopefully you have identified before you pay your supplier they are not registered for GST, ask them to re-issue the invoice without GST.  If you realise they are not registered after you pay, you can ask them to return the GST. If they won’t, it’s just an extra tax deduction for you. 

Always check the ABN register before paying new suppliers.