Completely agree with you, I have the same experience. Many times I feel like garbage, only for my numbers to be in the safe range 70-100, but if I eat, the symptoms will go away.
Every two hours all day I get really weak, fatigued, and dizzy and can't think clearly, etc. All the hypoglycemic symptoms we're all familiar with. My blood Sugar reading will be decent, but even so, until I eat, I'll get worse and worse.
The way I interpret it is that when time has passed and my actual blood Sugar reading is within safe ranges, I get the symptoms and extreme fatigue because my body is using its OWN energy, hormones, and nutrient stores to maintain and balance the blood Sugar level, keeping it decent, as opposed to the body using FOOD for fuels to keep it elevated and balanced.
With food your body will only expend digestive energy, but it'll get it all back once the food is digested and in the system.
Once the measure of energy gained from the food is expended, the whole body is outputting energy to maintain and balance the blood sugar, and since we're all sick, it is very straining on the body, so we get symptoms. Until we eat food to reduce the body's workload.
I eat a lot of meat and fat ALL day, with my last meal being within 10 minutes of laying down to sleep. I've been able to go longer in the mornings before I eat, but it's not without consequence. Consequences being what I described above.
EDIT: I should point out that I DO get true hypoglycemic numbers though. Low 60s, and possibly below 60, although I haven't had a chance to see numbers lower than that because either I don't have test strips off hand, or I'm so out of it that I can only function enough to eat immediately. I've also noticed that when I'm getting really bad symptoms and my blood sugar is low, my fingers refuse to bleed, even if I squeeze the lancet site all hard, so that also prevents me from seeing really low numbers.