To understand how to improve program or system performance, it helps to know what the ideal performance envelope is. If you get that wrong, no amount of profiling afterward will regain the lost performance.
Some useful rules are:
1. Brigg’s Second Law: “Every server computer sold today is a supercomputer.”
Corollary: “If your program isn’t fast enough, it’s your fault. Dig deeper.”
To understand this more deeply, you can read about the limits of computation, such as the Landauer limit. To summarize, since the goal of any program is to flip bits, the Landauer limit tells you what the minimum cost is (ie. maximum performance) and your program probably isn’t there yet. 
2a. Mike Busch, aerospace businessman:”The highest form of technology is measurement tools, because they have to be more accurate than the item being created.”
2b. “The ultimate goal in diagnostics is the direct measurement of the subject under test (think borescope or ICE.)”
3. Seymour Cray: “As long as we can make them smaller, we can make them faster,” meaning, “The speed of light is a limiting factor – the more local, the faster.”
Corollary: “each foot costs one nanosecond.”
Question for yourself: “what is the wire-speed (fastest possible) for this?”
4. Chip Salzenberg’s Rules for High Performance programming:
- don’t do it
- do it later
- let someone else do it.
Throughput vs Latency and Lock-Free vs Wait-Free
W: History of Supercomputing
Ultimate physical limits to computation
Limits on Fundamental Limits to Computation