The midwife may have been oversimplifying information in a way that ended up confusing you.
There is a common screening test that they do with the infant within a few days of birth that can test their hearing without needing observable feedback from the infant (i.e. the startle test or similar); in fact, there are two, the Otoacoustic Emission test (OAE) and the Auditory Brainstem Response test (ABR).
For my children, they used the OAE, which is a simple test that involves making small sounds in the baby's ear and detecting if the inner ear (cochlea) emits a particular sound in response to hearing a sound. This is possible for any age child, including a day or two old (mine both were tested at 1 day old).
What the midwife may have been referring to, though, is that this test has a very high false positive rate, particularly for children in certain birth circumstances - meaning children will appear to fail the test but actually have normal hearing. This happened for both of my children; my first had a borderline failure for one ear, which they retested at 3 months and he passed; the second had a cannot-test for one ear and a failure for the other, both passed a few months later again.
So, it's well possible that the midwife (or more likely a post-delivery nurse) in fact performed this test (if your baby was kept in a nursery and not with the mother, this could've happened without you fully understanding it, though I would've expected you at least sign off on doing the test). If that's the case, the explanation that 'it is hard to test' may have been intended to allay concerns about failing the test (and just poorly explained, particularly if you think a test was not performed).
All that said, I don't know anything about care in South Africa, but this seems to be standard practice in the US.