PowerShell: Duplicate file finder ( by File Hash)


There are plenty of software on the market to find duplicate files. Almost all of them list the duplicates and have you review and delete them ONE by ONE. So I decided to write my own PowerShell script to
* Find the duplicates by file hash
* move the duplicate files to the given location

With a single click, moving the duplicates to a directory was most easy way to deal with duplicates. (I select the files and delete them at once).

Here is the script (example is in the comment section). ENJOY!

<#
  ___                       ___  
 (o o)                     (o o) 
(  V  ) Duplicate Remover (  V  )
--m-m-----------------------m-m--

This script finds the duplicate files by
file hash and MOVE the duplicate files to
a given location. 

You can late review and delete all 
duplicate files from the given location.

Script written by: Anand, the Awesome

Parameters: 
Path : Directory Path of the files where do you want to check the duplicates
DuplicateFileMoveLocation: Directory Path where to move the duplicate files. If this
directory doesn't exist, it will be created.

Example: 
.\DuplicateFinder.ps1 -Path C:\Temp\Myfiles -DuplicateFileLocation C:\Temp\Myfiles\Duplicates

#>
param(
[Parameter(
Mandatory=$True,
ValueFromPipeline=$True,
ValueFromPipelineByPropertyName=$True
)]
[string[]]$Path,
$DuplicateFileMoveLocation = "")


Write-Host "Calculating number of files in $path..." 
$TotalCount = (Get-ChildItem $Path).Count
Write-Host $Path "have " $TotalCount " files."
Write-Host "Started finding the Duplicates..."
# Get all duplicate files
$DuplicatePaths = 
    Get-ChildItem $Path -File | 
    Get-FileHash |
    Group-Object -Property Hash |
    Where-Object -Property Count -gt 1 |
    ForEach-Object {
        Write-Host "Duplicated File: $($_.Group.path)" -ForegroundColor Yellow
        $_.Group.Path | Select-Object -First ($_.Count -1)
    }

Write-Host ($DuplicatePaths.Count) " Duplicate Files found. `n"
Write-Warning ("The Script found the {0} duplicate files out of {1} total. The duplicates will be moved to {2}." -f $DuplicatePaths.Count, $TotalCount,$DuplicateFileMoveLocation)
$answer = Read-Host -Prompt "Do you want to Proceed (Y or N)?"

if ($answer -eq 'y') {
    if ($DuplicateFileMoveLocation -ne "") { 
        # Create the duplicate file move directory if it does not exist.
        if ((Test-Path -Path $DuplicateFileMoveLocation) -eq $false) {New-Item -Path (Split-Path $DuplicateFileMoveLocation -Parent) -Name (Split-Path $DuplicateFileMoveLocation -Leaf) -ItemType "directory" *> $null}
        # Move the duplicates 
        $DuplicatePaths | ForEach-Object { 
            Write-Host "Moving $($_) to $DuplicateFileMoveLocation..." -ForegroundColor Red
            Move-Item -Path $_ -Destination $DuplicateFileMoveLocation 
        } 
    }
}
Write-Host "* * * Completed * * *" 
<#
End of the Script
#>

Office 365: How do convert Teams to an Office 365 Group?


I see there where plenty of sites/blogs explains how to convert an Office 365 group to a Teams. Super easy to change it using Teams client. I did not see any reference to other way around. If you find this blog, Good for you.

I see few users created “Teams” in Teams client. They wanted to use the Office 365 group mailbox for that Teams. With Client side settings, there is no options to do that at all. As an Exchange Administrator you can do it in PowerShell.

For every “Teams” created in Teams client, you will see an “Unified Group” aka “Office 365 Group” in Exchange. So all we have to do is change is unhide this Office 365 group for Exchange Outlook users. Here is how you do it (Make sure you connect to Exchange Online first) in PowerShell:

# Replace "TeamsName" with your name of the Teams in double quotes
Set-UnifiedGroup -Identity 'TeamsName' -HiddenFromExchangeClientsEnabled:$false -HiddenFromAddressListsEnabled:$false

After you set to False to both properties, The Teams will show up in Outlook under Groups in Mailbox tree if they are members. Give it plenty of time ( 30 mins ) to take effect of this setting change.

Did it help you? leave me a comment.

Exchange Online/OWA: Where is the HELL is “Send event invitations in iCalendar format” option?


Do you have IMAP/POP3 users? It amazes me to see the some stubborn users refused to use Microsoft Outlook on their Mac/Windows computers. They uses the limited featured IMAP client. Hey! Whatever makes you happy, I guess.

Are they looking for the option “Send event invitations in iCalendar format” in OWA settings? This options converts meeting requests into a iCalendar file attachment.

They are expecting to find the option under

OWA Settings >>> Mail >>> Accounts >>> POP and IMAP

OR

OWA Settings >>> Mail >>> Sync email

User-added image
I am sure this is the OLD OWA UI.

Instead they are seeing this in the new OWA UI.

Settings 
p Search settings 
General 
E Mail 
Calendar 
People 
View quick settings 
Crynpose and reply 
Junk email 
Customize actions 
Sync email 
Message handling 
Automatic rephes 
Retention policies 
S/MIME 
Sync email 
POP and IMAP 
POP optiom 
Let devices and apps use POP 
O Yes 
@ No 
POP 
Access 
MAP 
Server narne: outlookcfte365.com 
Port 993 
Encryption rnethod: TLS 
SMTP setting 
Server name: smtp.office365.com 
Port 587 
Encryption rnethod: STARTTLS
So, Where the HELL is “Send event invitations in iCalendar Format” option?

I have learned that user cannot change this settings anymore. BUT as an Exchange Administrator, YOU can change this setting behalf of the user. Here is the PowerShell Command (connect to Exchange Online first):

# Provide the identity as email address of the user
Set-CASMailbox -Identity StoneAgeIMAPuser@acme.com -ImapForceICalForCalendarRetrievalOption:$true -ImapUseProtocolDefaults:$false

Ta..da! Now that stone age IMAP user will start receiving future meeting requests in iCalendar file attachment. Say Hi! to them behalf of me.

Office 365: How to restore deleted OneDrive in PowerShell?


If an employee leaves the company, few managers wants the OneDrive content of the ex-employee. Few occasions it is business critical to take the documents from ex-employees’ OneDrive and re-share it again.

So how do you do that? I wrote the following script that does exactly that.

Before you run the script, make sure you have installed SharePoint Online module. Here is the help: Get started with the SharePoint Online Management Shell | Microsoft Docs

Note: Deleted OneDrive will be purged after 90 days. You can only restore the deleted OneDrive within 90 days of disabled date.

<#
    Script to 
    1. Restore the deleted OneDrive
    2. Assign the Restored OneDrive to manager or another employee


    IMPORTANT:  Replace the OneDrive URL for the variable $DeletedUserOneDriveURL and 
    admin URL in Connect-SPOService's URL parameter.

    Parameters:
    DeletedUsername : ex-employees' username or alias (Usually the name in the email address)
    NewUserEmail: Who needs to have access to the restored deleted OneDrive

     Example: 
     .\Restore-DeletedOneDrive -DeletedUsername JohnDoe -NewUserEmail Calvin.Hobbes@acme.com

#>

Param($DeletedUsername,$NewUserEmail)


Function Connect-ProofpointSharePoint
{
    Import-Module Microsoft.Online.SharePoint.PowerShell
    Connect-SPOService -Url https://acme-admin.sharepoint.com 
}

<#
    -*-*-*-*-*-*-* The Script Starts Here -*-*-*-*-*-*-*-*-*-*

    IMPORTANT:  Repalce the OneDrive URL for the variable $DeletedUserOneDriveURL

    Example OneDrive URL: https://acme-my.sharepoint.com/personal/username_acme_com
    In this example, "acme" is the company's Office 365 tag. The email address username@acme.com 
    is converted to  username_acme_come in the end of the URL. If you are in doubt, visit
    your OneDrive in Browser, Copy the URL from teh address bar and use it here.
    
#>

$DeletedUserOneDriveURL = "https://acme-my.sharepoint.com/personal/" + $DeletedUsername + "_acme_com"

# This command will ask for user credentials
"Connecting to SharePoint Online..."
Connect-ProofpointSharePoint

"Restoring the OneDrive: $DeletedUserOneDriveURL" 
Restore-SPODeletedSite -Identity $DeletedUserOneDriveURL

"Assigning $NewUseremail to the Deleted OneDrive"
Set-SPOUser -Site $DeletedUserOneDriveURL -LoginName $NewUserEmail -IsSiteCollectionAdmin $True

# -*-*-*- End of the Script -*-*-*-*-*-*

If you need to check if the “Deleted OneDrive” is already purged or not, check with these commands:

#Connect to SharePoint Online : Replace "acme" with your company's Office 365 tag

Connect-SPOService -Url https://acme-admin.sharepoint.com

# List all deleted sites includes OneDrive. Note "Days Remaining" the last column shows how many days left for purging

Get-SPODeletedSite -IncludePersonalSite

# List only the specific user

Get-SPODeletedSite -IncludePersonalSite | ? {$_.url -like '*JohnDoe*'}

I am sure this script and SharePoint commands will be helpful to you. Enjoy.

New kind of fresh HELL: Problem Ejecting USB Attached SCSI (UAS) Mass Storage Device. This device is currently in use


I ran into this issue of not able to eject an USB disk. Before you ask the question, I have set the disk for “Better Performance”. So I have to eject the disk before I remove it. (FYI, I would suggest to use “Better Performance” for bigger disks like 1TB or above to get most performance).

Now back to the issue, I was trying to eject the disk I am keep getting this error: “Windows is unable to stop the device” or “This device is currently in use”.

See the source image
See the source image

I freaking closed all the programs/windows, even killed explorer.exe from Task Manager. I tried logged out and logged back in.

If you run into this issue, I have a fix for you. Thank me later.

  1. Open Disk Management console (either run diskmgmt.msc in Run dialog – Windows Key + R, OR Right click on Start Menu and choose Disk Management).
  2. Find your disk in Disk Management console. Right click on the disk and choose Eject.

If you don’t see Eject, you may see “Offline” option. Then choose Offline, then remove the disk. Next time you connect this disk, you have to go to back to Disk Management and make it Online.

Tip- Disable/Enable web cam at will by PowerShell


Here is the one-line code to enable or disable the web cam when you want. Here is how it works.

I wrote a one line code and save it as a PowerShell (.ps1) file. Created a batch file to call the PowerShell script files to enable or disable the web cam.

Prerequisite

Open PowerShell and type (or copy/paste). This will display all of your Camera device(s).

Get-PnpDevice | ? {$PSItem.Class -eq 'Camera'} | fl FriendlyName,InstanceId
Here is the example:
E:> Get-PnpDevice | ? {$PSItem.Class -eq 'Camera'} | fl FriendlyName,InstanceId
 FriendlyName : Integrated Camera
 InstanceId   : USB\VID_04CA&PID_703C&MI_00\7&36726615&0&0000
 FriendlyName : S-YUE 8MP USB Camera
 InstanceId   : USB\VID_0BDA&PID_5075&MI_00\8&3E5D74C&0&0000

Now note down the Instance ID of the current camera (copy it to the clipboard).

Creating the PowerShell and Batch files

Disable Script

  1. PowerShell script file

Type the command below in a Notepad/Editor and replace the InstanceID you noted down above.

Disable-PnpDevice -InstanceId 'USB\VID_0BDA&PID_5075&MI_00\8&3E5D74C&0&0000' -Confirm:$false

Save the file as Disable-Camera.ps1.

2. Batch file

Type the command below in a Notepad/Editor and replace the path of your Powershell Scirpt file. Save the file as Disable-Camera.bat

PowerShell.exe -Command "C:\Temp\Disable-Camera.ps1"

Enable Script

  1. PowerShell script file

Type the command below in a Notepad/Editor and replace the InstanceID you noted down above.

Enable-PnpDevice -InstanceId 'USB\VID_0BDA&PID_5075&MI_00\8&3E5D74C&0&0000' -Confirm:$false

Save the file as Enable-Camera.ps1.

2. Batch file

Type the command below in a Notepad/Editor and replace the path of your Powershell Scirpt file. Save the file as Enable-Camera.bat

PowerShell.exe -Command "C:\Temp\Enable-Camera.ps1"

How to use it?

Here are my created files:

I made the shortcut for the Batch file on my desktop. When I needed, I right-click on the shortcut icons, select Run as Administrator.

To enable or disable the device, script has to run in elevated PowerShell. That’s why you have to run the shortcut Run as Administrator.

Better yet, I use Circle Dock to quickly launch programs. So I created two new icons for it.

How to sync any folder to OneDrive?


Do you want to sync/backup directories that out side of OneDrive sync’ed directory? For example, you have a directory at “D:\SDCard\PortableApps\PNotesPortable” and you want take a backup to OneDrive.

You have come to right place. This works for OneDrive (consumer) as well as OneDrive for Business. Here is how you do it.

Creating a junction for the directory D:\SDCard\PortableApps\PNotesPortable under OneDrive sync’ed location. That’s it.

  1. Open Command Prompt.
  2. Type this:
MKLink /J <OneDrive-Folder-Path> <Full path of the folder you want to sync>

Example:

C:\Users\Awesome>mklink /j "C:\Users\Awesome\OneDrive\Software\PNotes" "D:\SDCard\PortableApps\PNotesPortable"
Junction created for C:\Users\Awesome\OneDrive\Software\PNotes <<===>> D:\SDCard\PortableApps\PNotesPortable

Now you will see a Directory junction show up under C:\users\Awesome\OneDrive\Software. That’s all. Now OneDrive will sync this folder content to the cloud.

Nice Trick, hope you can use it yourself. Enjoy!

Exchange Online: Calendar Permissions – “Some Permissions cannot be displayed”


I needed remove few disabled users from the permission set in the Calendar. I got this error: “The security principal specified is already on the permission set”. To fix this error, I was using my other blog post: https://anandthearchitect.com/2018/06/29/get-mailboxfolderpermission-the-security-principal-specified-is-already-on-the-permission-set/.

Here is the wrinke. If you have Full Access permission to a mailbox, when you open the Calendar Properties dialog box and select Permissions in Outlook, you will see a message Some permissions cannot be displayed.

CalendarPermError

Fortunately This is documented in https://support.microsoft.com/en-us/office/user-experience-changes-for-sharing-a-calendar-in-outlook-5978620a-fe6c-422a-93b2-8f80e488fdec?ui=en-us&rs=en-us&ad=us.

The new Office software updates changed UI look ‘n feel of Sharing Calendar and managing permissions dialog box. They call it “much simpler user experience when sharing a calendar”. The side burn for this new UI update is that there are two sharing scenarios that are no longer supported.

  • Sharing a calendar if you have Full Access permission to your mailbox
  • Sharing a calendar if you have Author permission to a mailbox

Sure Microsoft. You had to make our life little harder and call it “much simpler user experience”.

The solution is:

1. Close Outlook

2. Add this Registry Key:

Registry key:
HKCU\Software\Microsoft\Office\16.0\Outlook\Options\Calendar\ShowLegacySharing

Type:
UX REG_DWORD

Value: 1

3. Open Outlook and try managing the Calendar permissions now.

Hope this helped you. Enjoy.

Exchange (OnPrem): How to make Emails from Internal Application servers are trusted/internal emails?


Usually Emails originated within Exchange Organization are considered “Internal” trusted emails. All other emails coming outside of Exchange organization are “External” emails and usually considered less trustworthy.

How to make the emails coming from internal application servers are trusted emails in On-Prem Exchange environment? Answer is understanding of how to setup the Exchange Receive Connector(s).

I would recommend you have separate receive connector with its own IP Address. What I mean is you assign additional IP address to the NIC on the Exchange Transport servers, specify this additional IP address in the Receive Connector to receive emails from Intranet servers and devices.

Open the Receive Connector properties window, go to Security. Enable Externally secured (for example, with IPsec) under Authentication settings, and enable Exchange Servers under Permission Groups as below.

Now for the keen people, the explanation for why we have to choose the above settings.

Externally Secured setting says the emails received to the connector are “External Authoritative” servers, so trust these emails. This is assuming the external servers are in physically controlled network and you know and trust the sources.

The ExternalAuthoritative authentication method requires the ExchangeServers permission group. This combination of authentication method and security group permits the resolution of anonymous sender email addresses for messages that are received through this connector. So this anonymous sender email address can have the domain names (e.g., DoNotReply@company.com) and Exchange will accept these emails.

Hope you had a clear idea after reading this blog. Leave me a “Hello” below.

Sources:
https://docs.microsoft.com/en-us/exchange/receive-connector-authentication-mechanisms-exchange-2013-help
https://docs.microsoft.com/en-us/Exchange/mail-flow/connectors/receive-connectors?view=exchserver-2019

Move Mailbox to OnPrem: “MigrationPermanentException: Cannot find a recipient that has mailbox GUID “


You are trying to move a mailbox from Exchange Online to Exchange OnPrem, and you get this error message:

“MigrationPermanentException: Cannot find a recipient that has mailbox GUID <GUID>”

MSKB Article explains why it happens and how to solve it. It involves note down the mailbox GUID from Exchange Online and modifying the mailbox GUID in OnPrem to the Remote Malibox. That is two different PowerShell sessions (one to Exchange online and another one to Exchagne OnPrem).

If you use my script to connect both Exchange Environments on the same PowerShell Window, you can fix the GUID in single command.

So here it is:

Set-OPRemoteMailbox -Identity <MailboxName> -ExchangeGuid (Get-EOMailbox -Identity <MailboxName>).ExchangeGuid.guid