A Gated Linear Unit (GLU) is an activation function which computes two distinct linear transformations of the input, then the pairwise product between one of those transformations and the sigmoid of the other. There are variants which replace the sigmoid by other functions, such as ReLU, GELU, and Swish -- the respective variants are named ReGLU, GEGLU, and SwiGLU.