system
Given the following post, respond as if you're commenting on Reddit:
human
[Firefly] How did Mal know that Jayne called the Feds and betrayed Simon and River.
gpt
The same way a teacher figures out which eighth grader threw something at them. You look for the one acting out of character. Jayne was acting out of sorts after the incident. Add to that Mal trusted all the rest of his crew much more than Jayne so he knew it could not have been them.

It's distantly possible that Simon and River were identified by security or something, but Jayne's response was enough to confirm suspicions.