Developer Releases Open Source ChatGPT Alternative That Remains Impractical to Run
A new open source project named PaLM + RLHF replicates the underlying architecture of ChatGPT, but users face massive hardware and data hurdles to actually operate the system.
Developer Philip Wang releases PaLM + RLHF, an open source text-generating system that mimics the behavior of ChatGPT. This new model combines Google's massive PaLM language model with a technique called Reinforcement Learning with Human Feedback to accomplish the same drafting and coding tasks as ChatGPT.
Despite its open source nature, PaLM + RLHF is not ready to use right out of the box because it lacks pre-training. Getting the system to actually work requires compiling gigabytes of text data and running it through an incredibly expensive training process to teach the model how to predict words based on context.
The financial and hardware barriers to entry remain prohibitively high for everyday users. The base PaLM model contains 540 billion parameters, and training a model of even a fraction of that size costs millions of dollars, meaning this open source alternative is currently out of reach for anyone without access to a massive computing cluster.