The Source for this principle is the Passage of "VeHiyyeetem N'keeyim [You shall be Clean]": Pesachim 13a
Rules
Choosing between preventing suspicion, and violating Gd's Honor by sinning: Niddah 13a
Where a community is committing a suspicious act, they are less likely to be suspected: Rosh HaShanah 24b
Performing acts which could look illegal, in an area where no one will observe: Shabbat 146b; Beitzah 9a
Preventing suspicion via public pronouncement explaining an acts permissibility: Menachot 40a
Preventing suspicion by writing an open letter: Menachot 40a-b
Preventing an act which could cause others to question someone's lineage: Niddah 13a
Preventing misunderstanding of the law by on-lookers: Keritot 24b
Preventing suspicion by performing an act in private: Ketuvot 60a [and see Tosafot ממעכן and Ran]
Cases in which one causes others to misunderstand a law
Rav Mari entered into a business deal with required a full transaction, but an onlooker thought he only made a verbal commitment, and emulated that: Bechorot 3b
Abbaye was an adam chashuv, and therefore he was stricter than necessary in certain practices lest people learn inappropriate conduct from him: Shabbat 142b
Rava was an adam chashuv, and therefore he was stricter than necessary in certain practices lest people learn inappropriate conduct from him: Shabbat 142b
Cases in which a person might be suspected of general impropriety
Praying behind a synagogue: Berachot 6b
Separating Dough-Tithe from dough which is technically exempt, because the King is to take it for his tithe: Pesachim 6a
Sitting on a marketplace bench on Shabbat: Pesachim 51a
Moving a ladder, which might appear to be preparation for fixing one's roof, on Yom Tov: Beitzah 9a-b
Eating the milk which is at the outer lip of a cow's stomach; the prepared form is called "Di'Aytera": Pesachim 51a
Suspicion of having idolatrous forms: Rosh HaShanah 24b
Suspicion of possessing idolatry if one purchases silver pieces with money and then discovers idolatry among them, so that when returning them the Jew receives money back and appears to be selling idolatry: Bechorot 13b
Leaving laundry out to dry on Shabbat: Shabbat 146b; Beitzah 9a
Hanging laundry to dry inside a Succah, where it will appear to be a canopy of invalid material: Succah 10b
The Overseer of the Communal Kitchen shouldn't buy the leftover food himself: Pesachim 13a, Bava Metzia 38a
Buying the Charge one is guarding, in a case where he is supposed to sell it due to rapid depreciation: Pesachim 13a
Showing the gold tables [which were bought with donations] to those who ascended to Jerusalem for Holidays, to show that there was no cheating in use of their Offerings: Pesachim 57a
Dealing with a partner upfront when dissolving an investing/managing partnership: Tosafot Bava Metzia 69b "ki"
Rav's care in arranging a business investment, lest it appear like prohibited ribbit, because of his status as an important person: Bava Metzia 73a
Requiring a pauper to use funds in the way stated by his benefactor, lest people suspect that the benefactor had not fulfilled his commitment: Bava Metzia 78b
Suspicion of wrongdoing which might lead others to also be lax
License for a person to eat kodashim when he says he is pure, even if others say he is impure - but only to eat them in private, because of Marit Ayin concerns: Keritot 12b [see Rashi there]