- [Home page](https://www.brimit.com/)
- [Blog](https://www.brimit.com/blog)
# Checking duplicate media items
[#How-to](https://www.brimit.com/blog?categories=#How-to) [DXP](https://www.brimit.com/blog?platforms=DXP)
[Evgeniy Triputko](https://www.brimit.com/blog/author?authors=Evgeniy%20Triputko) on January 24, 2024
In my project, I have a big media library with a lot of media. Sometimes content editors could load the same images and that was reflected in the size of the library. I decided to add a little feature for preventing similar images from loading.
The essence of my method is to create a hash for media items, save to a Solr index and add a processor for checking when the content editor loads a media file.
For the hash, I use System.Security.Cryptography.MD5. Firstly I created a method that converts Stream to MD5.
```
public static string StreamToMD5(Stream stream){ var md5 = MD5.Create(); Byte[] bytes = md5.ComputeHash(stream); var hash = md5.ComputeHash(bytes); return Convert.ToBase64String(hash);}
```
As you can see, I converted the hash to Base64. It is needed to simplify storage in Solr.
After that, I implemented a computed field that will keep the hash value.
```
public class MediaHashComputedField : IComputedIndexField { public string FieldName { get; set; } public string ReturnType { get; set; } public object ComputeFieldValue(IIndexable indexable) { var item = (Item)(indexable as SitecoreIndexableItem); if (item == null) { return null; } if (!item.HasBaseTemplate(Sitecore.TemplateIDs.UnversionedImage) && !item.HasBaseTemplate(Sitecore.TemplateIDs.VersionedImage)) { return null; } var mediaItem = new MediaItem(item); using (Stream stream = mediaItem.GetMediaStream()) { if (stream == null) { return null; } return MD5Helper.StreamToMD5(stream); } } }
```
Only Unversioned and Versioned images will have this hash in Solr. You can extend it for other media file types.
Connecting the computed field through Sitecore configuration.
```
DTCM.Feature.DuplicateMedia.ComputedFileds.MediaHashComputedField, DTCM.Feature.DuplicateMedia
```
The second step is creating a searcher that will have a method for searching images by hash.
```
public interface IMediaSearcher { IEnumerable GetMediaByHash(string hash); }
```
```
[Service(typeof(IMediaSearcher))]public class MediaSearcher : IMediaSearcher{ protected string IndexName => “sitecore_master_index” protected ID mediaLibrary => new ID("{3D6658D8-A0BF-4E75-B3E2-D050FABCF4E1}"); public IEnumerable GetMediaByHash(string hash) { if (string.IsNullOrWhiteSpace(hash)) return null; var index = ContentSearchManager.GetIndex(this.IndexName); using (var context = index.CreateSearchContext()) { var query = context.GetQueryable(); query = query.Where(i => i.Paths.Contains(mediaLibrary) && i.MediaHash == hash); var result = query.GetResults()?.Select(x => x.Document); return result; } }}
```
I use this searcher in a custom upload processor.
The custom upload processor looks like this:
```
public class CheckDuplicatesByHash : UploadProcessor{ //connect searcher through ServiceLocator, this searcher described above protected IMediaSearcher mediaSearcher => ServiceLocator.ServiceProvider.GetService(); protected HttpContextBase httpContextBase => (HttpContextBase)new HttpContextWrapper(HttpContext.Current); public void Process(UploadArgs args) { Assert.ArgumentNotNull((object)args, nameof(args)); if (args.Destination == UploadDestination.File) return; //content editors can load list files foreach (string file1 in (NameObjectCollectionBase)args.Files) { HttpPostedFile file2 = args.Files[file1]; //images place in archive if (!string.IsNullOrEmpty(file2.FileName)) { if (UploadProcessor.IsUnpack(args, file2)) { ZipReader zipReader = new ZipReader(file2.InputStream); try { foreach (ZipEntry entry in zipReader.Entries) { if (entry.Size == 0) { continue; } var hash = MD5Helper.StreamToMD5(entry.GetStream()); var searchMediaItems = this.mediaSearcher.GetMediaByHash(hash).ToList(); if (searchMediaItems.Any()) { string text = HttpUtility.HtmlEncode(file2.FileName + "/" + entry.Name); var duplicateItemNames = string.Join(", ",searchMediaItems.Select(i => i.Name)); var duplicateItemPath = string.Join(", ",searchMediaItems.Select(i => i.Path)); var errorText = $"The file \"{text}\" is duplicate sitecore media {duplicateItemNames} ({duplicateItemPath})."; if (this.httpContextBase.Request.Path.Contains("Upload Media/UploadMedia2.aspx")) { this.httpContextBase.Response.Write(this.ErrorMessageScript(errorText)); } args.ErrorText = errorText; args.AbortPipeline(); return; } } } finally { file2.InputStream.Position = 0L; } } //images are loading without archive else { var hash = MD5Helper.StreamToMD5(file2.InputStream); var searchMediaItems = this.mediaSearcher.GetMediaByHash(hash).ToList(); if (searchMediaItems.Any()) { string fileName = HttpUtility.HtmlEncode(file2.FileName); var duplicateItemNames = string.Join(", ", searchMediaItems.Select(i => i.Name)); var duplicateItemPath = string.Join(", ", searchMediaItems.Select(i => i.Path)); var errorText = $"The file \"{fileName}\" is duplicate sitecore media {duplicateItemNames} ({duplicateItemPath})."; if (this.httpContextBase.Request.Path.Contains("Upload Media/UploadMedia2.aspx")) { this.httpContextBase.Response.Write(this.ErrorMessageScript(errorText)); } args.ErrorText = errorText; args.AbortPipeline(); break; } } } } } //method for show alert with information about same image which sitecore has private string ErrorMessageScript(string message) { return $""; }}
```
This method is not elegant but it works with different loading types (through an archive, with list images, etc.)
Connecting the processor to Sitecore configuration.
```
```
**Demo**:
Uploading a file with the same image that I have in Sitecore.
###### Author
Evgeniy Triputko
Senior Sitecore Developer
[Evgeniy Triputko](https://www.brimit.com/blog/author?authors=Evgeniy%20Triputko)
###### More By Categories
[#Events](https://www.brimit.com/blog?categories=#Events) [#How-to](https://www.brimit.com/blog?categories=#How-to) [#News](https://www.brimit.com/blog?categories=#News) [#Guides](https://www.brimit.com/blog?categories=#Guides)
###### More by Platform
[Data and AI](https://www.brimit.com/blog?platforms=Data%20and%20AI) [DXP](https://www.brimit.com/blog?platforms=DXP) [E-commerce](https://www.brimit.com/blog?platforms=E-commerce) [Internet of Things](https://www.brimit.com/blog?platforms=Internet%20of%20Things) [Sales and marketing automation](https://www.brimit.com/blog?platforms=Sales%20and%20marketing%20automation)
#### More on Sitecore
#Guides #How-to DXP E-commerce
##### How Vercel Will Help You Save Effort When Deploying Sophisticated Sitecore Projects
Optimize and accelerate the development and deployment of complex multisite Sitecore projects.
Alexei Vershalovich on July 17, 2024
[How Vercel Will Help You Save Effort When Deploying Sophisticated Sitecore Projects](https://www.brimit.com/blog/how-vercel-will-help-you-save-effort-when-deploying-sophisticated-sitecore-projects)
#How-to DXP
##### Training Up Tomorrow's Sitecore MVPs: a Mentoring Success Story
How to participate in the Sitecore Mentor program and help younger colleagues jump-start a career in Sitecore development.
Sergey Baranov on October 2, 2023
[Training Up Tomorrow's Sitecore MVPs: a Mentoring Success Story](https://www.brimit.com/blog/training-up-tomorrows-sitecore-mvps)
#How-to DXP E-commerce
##### Going Headless. Part 2: When a Headless CMS Is Your Best Bet (if you have Sitecore)
Discover how a headless CMS can benefit organizations that use Sitecore.
Daniil Raschupkin, Palina Trokhautsava on September 15, 2022
[Going Headless. Part 2: When a Headless CMS Is Your Best Bet (if you have Sitecore)](https://www.brimit.com/blog/going-headless-part-2-when-a-headless-cms-is-your-best-bet-if-you-have-sitecore)