So, in theory if you had a vacuum pump and put the vacuum chamber in a freezer, would that just be a much slower freeze dryer?

The description you have is nearly identical to the freeze dryer reviewed. The one feature missing from what you’re describing and the commercial product reviewed was the commercial product had warmers for the trays to speed up sublimation. Besides the tray warmers, the freeze dryer was a vacuum chamber wrapped in a freezer.