If you are using unpublished material from libraries' special collections or archives--whether you physically accessed the materials in a reading room, or accessed the collection online--you may need to consider website terms of use agreements, or contracts you signed (or clicked through online) with the institution. This is because, irrespective of whether materials held by archives or in libraries' special collections are protected by copyright, you may have entered into an agreement dictating whether you can publish from the works.
Why would some libraries and archives restrict your ability to publish from works in their collections? They often sign agreements with donors that, themselves, restrict reuse of the records being contributed. For instance, a donor of unpublished personal letters might--as a condition of donation--restrict use of the letters to researchers in a reading room, and prohibit publication or digitization. The archives may pass this condition on to you with a "Terms of Use" (or equivalent) agreement. In exchange for the archives granting you access to the correspondence, you may waive your right to publish excerpts from the letters as fair use, since contract law trumps copyright.
When using materials from databases, websites, archives or special collections, read carefully any agreement or terms of use that you are asked to enter into. Most databases, sites, libraries, and archives have pages that explain their permissions policies. For instance, UC Berkeley's Bancroft Library has a permissions page here.
If the site or archive's terms of use do not let you reproduce the materials, inquire about whether a waiver is possible, or seek additional information from the archive about securing rights to publish.
Privacy rights most often arise if you are seeking to use third party content like correspondence, diaries, and images that contain personal information about or pictures of particular people.
In addition to a number of federal statutes that protect against disclosure of various types of personal information (e.g. FERPA (student information) and HIPAA (health information)), there are also state laws governing privacy.
State privacy laws make intrusions of privacy a tort (i.e. a wrongful act). The four typical types of intrusions that state laws protect against are: intrusion upon seclusion; public disclosure of private facts; painting someone in a false light; appropriation of name or likeness.
You may not need to dig too deep. There are important limitations on privacy rights in the context of your scholarship.
First, privacy rights expire at death--meaning, you can't be liable for disclosing private facts about a person no longer living.
Second, there are typically two other important defenses to claims for invasion of privacy: newsworthiness, and permission. If the material you wish to include reveals private facts that are "newsworthy" (of public interest or concern--which your scholarship may be), or the person who is the subject of the information has given you permission to publish (which you may have obtained), then an invasion of privacy claim should not be sustainable. For more on newsworthiness and permission exceptions, exploring the Digital Media Law Project out of Harvard University's Berkman Klein Center should prove helpful. It is no longer being updated, but contains very useful explanations.