Activity trackers are useful for seeing how consistent you're being, but not very accurate for gauging energy expenditure (
they have been studied and there's lots of articles about said studies showing how off they can be). If you track your weight and it's currently stable, then your caloric intake is sufficient and eating more is just going to lead to weight gain, although maintenance calories exist within and range and it can be useful to see what the upper end of this range is. 4000 calories is way higher than most people would need to weight-stable; just off the top of my head, you'd have to be an extremely active 100kg male to warrant that level of intake.
If you're tired all the time, my first port of call would be to look at if you're meeting your caloric needs but food choices overall could be more healthful nutritionally in terms of macro and micro nutrition and if you're getting enough sleep. Doing heavy work all the time without giving your body much time to recover is inevitability going to lead to some form of fatigue as well - more food can't make up for a lack of rest.