Second, the Federal Reserve is not a cabal, let alone a shadowy one.

Third, it's existence is to ameliorate recessions. Your great-great grandparents, if in the US at the time, were very much in favor of it.

Fourth, the US doesn't allow for sovereign banks, so a private Federal Reserve is already the classical liberal/libertarian solution (anarcho-capitalists are typically the ones calling for abolishing the Federal Reserve/returning to commodity as a standard).

Fifth, a lender of last resort is a fantastic backstop allowing more investment and risk taking, which has pushed the US' investment dominance for over a century.