Shell Trap is a decreased priority move. The user sets up a trap at the beginning of the turn, then if it gets hit by an opponent's physical move within the same turn, it will use Shell Trap immediately after that move. Otherwise, when the user's scheduled turn happens at -3 priority, Shell Trap will fail at that time. If the user is affected by Powder, Powder will trigger when the user of Shell Trap is hit by a physical attack.
Encore will fail if the target has an active Shell Trap in waiting.
Shell Trap will not be triggered from the Pokémon being hit by an ally's move, having its substitute take a hit, being hit by a move that's receiving a bonus from Sheer Force, being hit after its own turn has already passed (such as by Dragon Tail or another move with priority lower than -3), or being hit by a move that causes it to faint.
Due to a glitch, if a Pokémon's use of Shell Trap is preempted by Encore to lock it into another move instead, and the Pokémon is then hit by a physical attack on any turn before Encore ends, the Pokémon's next use of Shell Trap will skip the step of setting a trap, and the move will go off successfully at its normal priority regardless of whether it was hit in the current turn.