Improve logging on Email Signature Server for mail sending
We have improved our internal logging for message sending. This release is not changing the Email Signature Server's behavior.
Avoid throwing exception when trying to track a message with null id
When devices such as printers are sending emails through the Exchange Server and the sender addresses are not filtered out with specific rules, the Email Signature Server was dropping the messages before identifying them as being device messages. Now, they should be correctly identified and relayed.
Distribution lists blocked in pending after upgrading SmtpServer library
Whenever a user is sending an email to a distribution list, the email is received by the Email Signature Server from Exchange already expanded. This means that instead of having one recipient, the email has a recipient list containing all members of that distribution list. At the same time, Exchange might split the mail into multiple transactions of at most 1000 recipients, based on the user's recipients limit and mail sending reputation.
To avoid having the user blocked by Exchange because of reaching sending limits, we were splitting each transaction into multiple of at most 100 recipients. After performance improvements, other Exchange limits were hit and we increased the batch size of recipients to 300. Also, for better stability in the SMTP connection, we are now having a retry mechanism on the SMTP connection when sending the mails back to Exchange.
Investigate regex timeout error log
We have modified the regular expressions to be more efficient, which will help alleviate this problem, although perhaps not resolve it fully.
Upgrade Email Signature Server infrastucture
In our continuous efforts to improve the stability and performance of the Email Signature Server, we have updated our core components responsible for processing and sending the emails.
This change does not affect any of the email signature server functionalities, but it improves the performance and reduces the CPU and memory resources consumed by it.
Email signature not attached to Blackberry Mail app reply emails
When using Blackberry Mail app and replying/forwarding to an email, the signature was either inserted at the end of the conversation or it wasn't inserted at all if the email was already containing a Templafy signature from a previous mail in the thread. This was happening because we weren't able to identify the Reply/Forward divider for this scenario. We are now also supporting it.
Authenticated Results Chain (ARC) authentication fails on email sent through Email Signature Server
The ARC (Authenticated Results Chain) authentication on emails sent through the Email Signature Server (ESS) was failing due to the changed body resulted by attaching signature. We are now removing the ARC headers after attaching the signature, so that the entire ARC-message-signature is recomputed based on the content containing the attached email signature. This is not applied if the ARC-Authentication-Result is already failed when reaches ESS.
DKIM failures when processed by Email Signature Server, receiving systems flagging as spam
When DKIM is enabled as a security feature in Office 365, a message header is added to the email that is containing a checksum for the HTML body. Because we are changing the HTML body, the checksum doesn't match when it is verified on the recipient side. Some email clients are detecting this as spam. We are now removing the DKIM header received on the Email Signature Server in order to trigger its recreation on Office 365 after the email was sent back from us. This is assuring that the DKIM body checksum is computed according to the updated HTML with signature.
Email Signature Server stops sending batch emails immediately if one fails
When the number of recipients is larger than 100, we are splitting the list into batches of at most 100 recipients. This is done because we don't want to reach the Office 365 recipient limits for Distribution Lists. However, when one recipient was failing in one of the first batches, an exception was immediately thrown and the next batches were not processed anymore. Now we are waiting for all batches to be processed and so the mail to be sent to all recipients, and only afterward we are returning the exception.
Failed to apply signature: Object reference not set to an instance of an object
We identified an issue with processing "attachment only" e-mails e.g. sent from printers. This led to an error and no signature was attached. These e-mails types are now processed correctly in this release.
Plain text messages are processed when no signature is available
When no signature was available for a user, and the e-mail was in plain text, it was still trying to update the email message and the item in the sent folder. This led to unnecessary processing and logging.
We are now correctly identifying and handling this scenario.
Exception thrown on monitoring when multiple threads access at the same time
In a previous release we introduced a more detailed message monitoring keeping count of the processed messages. This had an issue that triggered an exception if two messages happen to log the data at the exactly same time. This is now resolved.
Emails not being processed due to wrong email address format
In cases where one of the recipients had an invalid address format in the email header the email signature server would fail sending the email. However, the email address format is often still valid in the email envelope.
In this update we make sure that email sending is not affected by the header parsing. If the header contains invalid entries this is now logged and the email is sent using the information in the envelope.
Signature not added to a limited set of emails
For a limited subset of emails we were unable to locate where to insert the signature. These were sent on without getting a signature attached. These emails are now detected and get a signature attached.
Add observability and health check
To prepare for adding further pod health check probes we have added more tracking to know when an email (messageID) is started and finished being processed. This can now be queried for in Application Insights.
0.2.0.16XX- (Terraform/Kubernetes update only)
Important: The Terraform scripts have been updated and the cluster configuration that is built has changed.
Node and Pod auto-scaling
Before this change we needed to decide upfront a set fixed count of nodes (virtual machines). Now we can specify a range of nodes and based on the current usage the Azure Kubernetes service will automatically create a new node and decrease latter when the usage drops. This node autoscaling works in tandem with the Kubernetes Horizontal Pod Autoscaler. If Kubernetes sees that the new pods needed based on usage will not fit on the current number of nodes it will create a new node up to the max specified. This pod scaling is done based on CPU and memory consumption.
Email stuck in sending loop
In some rare cases email parsing resulted in high CPU usage of the cluster. This was triggered by emails having invalid separators between new and reply sections. These emails are now detected and passed back to Exchange server without adding a signature.
Dynamic IP Validation
This feature solves the problem regarding the validation of the sender IP address within Email Signature Server. The list of IP ranges from spf.protection.outlook.com was not always being updated, causing rejection of valid IP addresses. Email signature server now validates the incoming IP address dynamically. Templafy is now running a daily job to get the valid Exchange IP ranges from Microsoft and synchronize them to every Tenant's Email Signature Server blob storage.
Improve exception handling during message processing
Previously external factors like low memory, errors in SMTP communication, or other factors related to this and our infrastructure could result in unhandled error propagating to Exchange Online. These unhandled errors resulted in failures to the email delivery, and the senders received a Non Delivery Receipt.
This release ensures that all errors are properly handled and that the corresponding SMTP code is returned to Exchange Online, signaling to Exchange to retry the sending of the email message and not to fail.
Recipient addresses with UTF8/non-ASCII characters in the local-part are rejected
Our Email Signature Server was experiencing issues in handling emails that were sent to addresses containing special characters. We identified the problem in the mail sending component, reported it, and then upgraded the Email Signature Server with the fixed version of the component.
Skip adding signatures to RTF emails containing email attachments
When sending emails containing a Rich Text Format (RTF) email as attachments (attaching another email in one email) or when sending email attachments in RTF emails we observed, in some cases, that the email is received with a corrupted attachment. This happens when the email is sent inside the organization and rich-text-format usage is allowed. To avoid attachment corruption we decided to temporarily skip these emails from processing and therefore we don't apply a signature in these cases.
As a result, we are not applying the signature if the following 2 conditions are met:
- An email is sent containing another Rich Text Format email as an attachment
- Rich-text-format in Exchange is allowed (automatic TNEF conversion by Exchange is not enabled)
When sending an email inside the organization Exchange is not using TNEF conversion on the entire message and the email attachment gets corrupted during our processing.
Because RTF and TNEF are proprietary languages of Microsoft, we decided to temporarily skip the signature processing of these messages until we find a way to properly recreate the logic Exchange uses when TNEF conversion is enabled.
Email sending timing out after MailKit upgrade
After upgrading to the latest version of the Mailkit component we found in testing that on rare occasions timeouts were occurring when sending emails. This was caused by the new version of Mailkit introducing a different way of setting timeouts, which meant that on connections(Network stream read/write) where we didn't have a timeout before we suddenly had one. The solution was to make sure the timeout values are the same as prior to the MailKit upgrade.
Relay original email if EWS call fails (for RTF)
When sending RTF emails the Email Signature Server asks Exchange Web Service for an Html copy of it and uses that to attach the signature. This also means that if Exchange Web Service is unable to return an HTML copy of the email for some reason we fail to send the email and the user receives a Non deliver receipt.
This behavior is now changed, so in case Exchange Web Service fails then we don't fail but resend the original email without the signature. Therefore making sure that the user always gets the mail delivered.
Update MailKit library
In our continuous efforts to improve the stability and performance of the Email Signature Server we have updated one of our core components responsible for processing and sending the emails.
This change does not affect any of the email signature server functionalities.
Handle the case where MessageId is null or empty
During testing we encountered situations where we received messages without a proper MessageId set. In this case, our application would fail, since it is not possible to process a mail without a MessageId. Now in such a case, instead of failing, we simply relay the message to exchange.
Don't pass mime message (containing attachments) when updating Sent Items
After the email signature is added to the email message and the message is sent back to the Exchange Server, the Email signature server also updates the item int the sent items folder with the signature. In this process we were passing the entire email message. We identified that this affected the performance of our Email Signature Server. We have now reduced this to sending the minimum of information required for this process.
No functionality was changed as a result of this improvement.
Optimize tracing lookup - Add configuration flag
Message Tracing was enabled by the presence of a trace_config.json file in the blob storage. To detect the presence of this file the application made a check every 5 minutes for the file in order for Message Tracing to be enabled. This resulted in an unnecessary processing overhead. To optimize this we have introduced a new flag to the configuration. By default this Message Tracing feature is disabled. To enabled Message Tracing it is now necessary for the enableTracing flag to be set to true and the presence of the trace config.json file in the blob storage.
We identified a problem with the caching component which caused delays in email message processing. We have replaced this component with MemoryCache, a library provided by Microsoft.
No functionality was changed as a result of this improvement.
0.2.0.13XX - (Terraform/Kubernetes update only)
Important: For the 0.2.0.XXXX series the Terraform scripts have been updated and the cluster configuration that is built has changed.
Enhance cluster configuration
After analysing the resource usage of our Email Signature Server we decided to adjust the default configurations created by the Terraform script:
- Use Virtual machine scale sets.
Microsoft recently introduced this new functionality, its biggest advantage is the possibility to configure Node Auto Scaling. The Email Signature Server does not support yet node auto scaling but this release is a prerequisite to support it in an upcoming release.
- Default Virtual Machine type changed Compute intensive F series.
We changed the default Virtual Machine type, moving away from the Generic D type machine and using the Compute intensive F series, which are the recommended ones for batch processing. They have a better CPU/Memory ratio (1 machine having 4 CPU and 8 Gb of memory, versus 2 CPU and 7.5 Gb of memory for the Generic D type). The default setup now consists of 24 pods running on 3 nodes.
- Update CPU and memory requirements and limits.
As part of fine tuning the resource usages in Kubernetes new pods will be allocated if 300Mb of memory and 300 Mi (0,3 of a CPU) is free and it will be allowed to consume 550 Mi (0.55 CPU) and 800 Mb.
- Update from Basic to Standard Load Balancer.
The Standard Load Balancer is now the recommended configuration which comes with a guaranteed SLA from Microsoft.
RemoteEndpoint is empty
Since the March 17th release of version 0.2.0.1238 incoming endpoints were being resolved to address 0.0.0.0. This did not affect the functionality of the Email Signature Server itself but did have an effect on the log entries produced. The RemoteEndpoint field in the EmailSignatureServer table did not contain the actual sender IP address. This has been corrected and RemoteEndpoint entries are now logged correctly.
Validate sender with key
To enhance the security level of our Email Signature Server component, we have added a new feature to validate the authenticity of the sender.
With this feature, the Email Signature Server only processes emails that are sent by an Exchange Online server of the customer. Even if an attacker is able to send emails using the customer's domain with another Exchange Online, we will recognize the unauthorized emails and reject them.
This feature is disabled by default since it requires the generation of a new security key which needs to be provided as an environment variable for the application.
To successfully enable this feature, the following steps are required (in this order):
Configure Exchange Online
Configure Exchange Online to add a custom header called Templafy-EmailSignatureServer-Secret that holds the secret as value in each email forwarded to the Email Signature Server.
Configure Email signature Server
Add an environment variable called templafyEmailSignatureServerSecret to the Email Signature Server in Kubernetes with the same secret as value.
When this feature is enabled, by adding the templafyEmailSignatureServerSecret environment variable, the Email Signature Server will reject all emails without the required secret in their headers. When an email is sent without this header, the sender will receive a Non Delivery Receipt with the following error message: 551 Unauthorized SMTP server. The same error message will appear in the Exchange Online message trace.
Upgrade Email Signature Server to .NET 3.1
We upgraded the target framework for our email signature server from .NET Core 2.2, to .NET Core 3.1. The functionality of the email signature server was not changed. This was required because Microsoft dropped support for .Net Core 2.2.
Terraform scripts; Add kubernetes_version to input.auto.tfvars
We currently use a terraform script to install the Email Signature Server in Kubernetes in self hosted environments. Until now the version of the Azure Kubernetes Service was hard coded into the terraform script, this didn't allow to set it to the latest supported version. The version parameter is now exposed as an input variable, giving us and customer the opportunity to update it at installation time.
This change does not have a behavior impact on the Email Signature Server.
Invalid content id format
We have discovered a scenario which resulted in an email not having a signature attached. This affected a small subset of emails sent inside the organization.
The signature was not attached in case a user received an email with an attachment from inside the organization and later on he/she would forward the email also inside organization. If any of the recipients or senders were outside of the organization then this would not happen. This has now been fixed.
Handling Distribution lists in Email signature Server
Our current email signature server solution currently supports sending emails to distribution lists. However it had an issue when the size of the distribution list exceeded the size of the maximum recipient list, which has the default value of 500 in Exchange Online. An alternate solution has been implemented and now we support distribution lists of any size.
Upgrade VM size in terraform script
The Vm Size was set to D1_V2, which is only 1vCpu and the recomended one now is 2 vCpu, si it was upgraded to DS2_V2.
Refine powershell script that creates certificates for Email signature server installation
Until now an admin needed to do several manual steps in order to generate a self signed certificate and export the certificate and related data into different formats. This powershell script automates this and all the admin needs to do is run it, enter a password and all the necessary output will be generated in the right format.
Failure inserting email signature for NullReferenceException
In a rare case when sending an email with a default client signature from a mobile device, a signature would not get applied. This was because of an unexpected HTML structure in the email when looking for the default client signature.
The structure of these emails can now be handled correctly.
Image gets removed from body when email has image just before default signature
When replacing a default client signature in an email there was a possibility that an image placed just before the signature would also be replaced. This has now been corrected so no visible element will be removed as part of replacing default client signatures.
Correctly detect emails sent from devices that do not have a mailbox
We are failing to update sent items for emails sent from devices, such as scanners. This is because the email address from which they are sent is not linked to a mailbox. We are now handling this case and skipping the update in Sent Folder process if no Mailbox is found for the sender email address.
Log Analytics workspace name must be unique
When creating a Email Signature Server environment with Terraform the log analytics workspace had a default name. This name is globally unique so this default name would not work in future setups. The log analytics workspace name is now part of the variables a tenant needs to configure.
Automate Email signature installation by using Terraform
We have introduced infrastructure as code (IAC) by using terraform with the aim of:
- simplify and reduce our setup process
- store the setup steps in code, with source control to provide history of the changes
- replicate a complete environment any number of times
Ensure default signatures are replaced without leaving up additional space
The problem we were facing is that if an email was sent from a mobile device and the email contained default device signature then upon replacing the default device signature with the Templafy generated one this one would be incorrectly placed one line bellow. Also if the user manually added multiple empty lines before the default device signature then this would be preserved, rendering the email signature only after these. With this improvement we make sure that in case an email has a default device signature then when we replace it with the Templafy signature this is placed correctly without having extra white spaces before.
Email Signature Server message visualizer
Note: This tool is only internal and is so far only available for developers of Email Signature Server. When investigating how an email will get a signature applied it can be cumbersome to process an email in the Email Signature Server to see the result. To make it easier to investigate where email signatures will be inserted and how the email was processed we can now use a visualizer tool. The tool can read files in EML, MSG, and HTML format and display the original and processed email side-by-side, among other information about the processing of the message.
Fail to retrieve email from exchange when trying to update sent item for read receipts emails
When accepting to send a read receipt for an email in outlook the Email Signature Server would not recognize it as an auto generated message and would apply a signature to the read receipt. This would also cause an error when trying to apply a signature to the sent item, which wouldn't exist in the senders mailbox. These read receipts are now correctly identified as auto generated messages and no signature will be applied.
Extend the logging to explicitly state Exchange Status Code returned
When an email would fail to send it was unclear which SMTP status code would be returned to Exchange. The returned SMTP status code will now be logged when an Email fails to send, which makes it easier to determine how Exchange will handle the email afterwards.
No reason logged when failing to fetch access token for the App Registration
In case the process of retrieving an access token to our app registration failed we would only get a generic error therefore not being able to see the real reason of the failure. This change logs the specific error code returned providing the details of the underlying exception.
Migrate to Email signature server to .Net Core 2.2.1
Currently our Email Signature Server component runs on .Net Core 2.0. Several performance issues that we experienced are linked back to this version of .Net Core so in order to solve them we decided to upgrade to the newest stable version, which is .Net Core 2.2.1. The change required us to update all our project files and also to use a new base image for our docker container. This change did not require any code updates or configuration changes in our orchestration framework. So the new version can be released to the rest of the customers without requesting them to do any changes to their hosting environment.
Auto-generated emails do not contain a valid sent date
In some cases an email will not contain a sent date. This is determined by the client that sends the email. We would previously not apply any signature to these emails, but with this change we will now use the current date as sent date instead of failing. This means that if an email is sent without sent date we will apply the signature that is valid at the time of processing the email, if a valid signature exists.
Obfuscate email addresses in table storage and slack integration for Email Signature Server
To be compliant with GDPR the Email Signature Server will now only save information about emails with obfuscated email addresses. This includes all log messages in table storage, slack, and all tracking information. Emails will be obfuscated with the same obfuscation format as website so they have the form "email@example.com".
Replacing default signatures can remove message when sent from Windows 10 Mail Client
When replacing the default email signature from Mail for Windows 10, part of the messages could accidentally be interpreted as part of the default signature. In such a case the default signature was not correctly identified and would result in the message being clipped. This has not been a production issue, as this particular email client is not in use, however, the issue has now been resolved.
Emails sent as RTF to external mail will not get signature applied
If a user without Outlook addins would send an email as RTF to a recipient outside the organisation there would be no signature applied by the Email Signature Server. This behavior is now changed so a signature will be applied when a user not using addins are sending emails as RTF to recipients outside organisation.
Email Signature Server monitoring doesn't log processing times
The Email Signature Server was no longer logging the processing time of emails since late November. This is now corrected so processing times for emails are now being logged again.
Ensure email signatures are only added to emails that do not contain a Templafy email signature already
This is extending our efforts on making sure that the email sent has a signature attached. Even if an email is sent from outlook with the VSTO add-ins enabled we check the content of the email to make sure that it contains a valid email signature. At the moment we are able to do this for email having message format of type HTML, which is the most common one. For plain text and RTF emails, we still rely on checking a header field value, which indicates if the VSTO add-ins were enabled when the email was sent. This story includes changes both for website and email signature server, the VSTO add-ins are not impacted. The website changes only contain the HTML template change on which the email signature generation is based. The customers are not going to get an email signature updated notification from VSTO.
Email with multiple HTML TextParts throws error
When sending emails from iPhone mail client with zip attachments the email will also include a HTML file. This file could be misinterpreted as a HTML body which caused an error when applying a signature. We will no longer now interpret HTML attachments as part of the email body.
Notify errors on Slack from Email Signature Server
This feature is the second step related to monitoring the Email Signature Server. In this feature, we focused on getting real time notifications if errors occur on our Email Signature server. We want to make these errors easier to address by making them more visible. We now stream these errors to a dedicated Slack channel: #log-emailserver-prod. Example errors are: to not being able to send an email, or not being able to update the sent item in the sender mailbox. For the moment, this feature is available only for Templafy and Templafy-hosted tenants (currently we have 2 partners).
Email Signature Server monitoring of events and run-time exceptions
The Email Signature Server for Office365 is supposed to process all emails sent from a tenant. In this story we take care of storing tracking and monitoring data for every email message processed. (Eg. UserIds, anonymized email address of the sender, type of email - new or reply/forward -, successful insertion, time used for processing the email and apply the signature, email signature server version ...). We collect useful data on the usage and performance of our service, but this story sets the grounds for future analytics work that we will eventually present to our admin users.
Email signature added in the wrong place for reply mail and not added following reply emails
When sending from a Sony Android device with default mail client the signature would either be placed incorrectly or would not be inserted at all. We are now able to handle this client and place the signature correctly.
Prevent invalid connections to Email Signature server
We had an issue where connections from non-Exchange Online servers could be made to the Email Signature Server. Although these connections wouldn't be able to send messages though Exchange, the Email Signature Server would still allow them to pass through. In order to avoid this, we have now implemented IP filtering based on the Exchange Online IP-addresses provided by Microsoft, meaning that any connections from a non-Exchange Online will now be rejected.
Make listening port configurable for email signature server
When testing Email Signature Server there are some difficulties always using port 25, like being unable to connect to that port from some locations. It is now possible to configure Email Signature Server to listen to another port than just 25.
Updating sent items fails in some cases
Updating an email in sent items would sometimes fail and the failure seems to correlate with the Email Signature Server having performance issues. The procedure of updating the sent items have been changed to avoid occurrence of this failure.
Ensure email sender validation in Email Signature Server
The connection between the Exchange Server and our Email Signature Server is done by IP. One malicious user, knowing our IP address, may directly connect to the Email Signature Server and send emails. This could lead to spoofing and impersonation attacks. For example, one could send an email as firstname.lastname@example.org, the email will be processed and will be sent back to Exchange Server as it was an email that came from it and should return. By this story, we ensure that the spoofing attacks and impersonation are not possible by using validation mechanisms. We now do not allow sending outside organization that does not contain a valid DomainKeys Identified Mail (DKIM) signature. DKIM validation is one method used for spoofing email prevention. The implementation requires the Office365 customer to have DKIM signatures enabled on all the domain registered (by following the steps provided https://docs.microsoft.com/en-us/office365/securitycompliance/use-dkim-to-validate-outbound-email ). DKIM verification is enabled by default but can be disabled by setting an environment variable in the Email Signature Server's Kubernetes setup. Exchange server is adding a DKIM Signature to all outside organization emails, that is validated by us. In case there is no DKIM signature or the signature is not valid, the email is considered spoofing and is blocked from sending. Emails inside the organization do not contain DKIM signature and are not put under validation. In the case of "Sent on behalf of" emails, the following rules are applied: Email Signature of the one the email is sent on behalf of will be applied Email is saved and updated in the sent folder of the sender (the one that sends on behalf of someone else)
Log insertion of signature when using plain text message handler
Until now it was not logged when a signature was inserted in a plain text email. This made it hard to see if a signature was inserted or not when troubleshooting. It will now be written to the log as information when a signature is added to a plain text email message.
Make it possible to disable EWS calls for load tests
In order to run performance tests we needed an option to disable web service calls to Exchange (Exchange Web Service). Disabling this will result in sent items not being updated and RTF emails will not get signatures added.
Set TLS 1.2 support in SmtpServer
In order to comply with the latest security requirements TLS was updated for Email Signature Server so TLS 1.2 is the default security setting.
Add StyleCop to Email Signature Server
To ensure the coding style is consistent across the entire solution, the Templafy.StyleCop package is added to make sure the rules are enforced.
Enable batch writing of log messages
We wrote log messages immediately, which resulted in some overhead since logs were written one by one. This has been changed to a more efficient approach, so we now write log messages in batches.
"Reply All" from sent items sets signature at the bottom
When using "Reply All" on an email in sent items in some edge cases the signature would be placed incorrectly. Signatures will now be placed correctly when using "Reply All" on emails in sent items.
Concurrency issue when two emails load default signatures at the same time
An error could occur in case two emails were being handled at the exact same time when default signatures needed to be loaded. This error would result in signatures not being added to the email. We now handle concurrent emails correctly in the case default signatures needs to be loaded.
Email Signature added 2 times, one before and one after the actual content
In some cases Email Signature Server would add a signature to the top of the email, even though a signature was already added by the Add-In. This situation could occur when creating a draft in Outlook Desktop with a signature inserted and sending it through Outlook Online. Email Signature Server will no longer insert a signature to the top when sending an email from Outlook Online which was created in Outlook Desktop with an Add In Signature already inserted.
Reply to certain email puts signature at the bottom
In some edge cases when replying to emails from outside the organisation, the signature would be placed incorrectly at the bottom of the email. Replying to these emails will no longer result in the signature being inserted incorrectly.
Default signature visible when sending with Outlook for Android Closed Default signatures was not being removed from emails sent with Outlook for Android. Outlook for Android removed the HTML tag that was used in inserted default signatures. We searched for that specific tag so, after this update, we were not detecting it anymore. This is now fixed by more dynamically identifying the default client signatures.
Add headers to emails processed by Templafy Email Signature server
Until now all emails sent by Outlook with the VSTO addins enabled were not forwarded to our Email Signature Server since they contained a special header. However in the future for analysis purposes we would require that all emails pass through our email signature server. To achieve that we changed the rule in exchange so that emails coming from Outlook with VSTO addins enabled will be forwarded to the email signature server and there they will get a new header called: Templafy-EmailSignatureServer-Processed: true.
Make sure we send the email
Until now if an error occurred in processing the message or when trying to update the sent items then it could theoretically happen that we don't send the email. We revised and changed so as long as we can retrieve an email we make sure we send it no matter what other components might fail.
Improve logging on configuration failure
In case there was a configuration error when starting up the email signature server we would only get a generic error message and we wouldn't be able to easily deduct which setting is wrong. Now we specify in the error message the exact details of the faulty setting.
Log message identification parameters in case message sending fails
When an error occured during a message processing we write an entry to our log repository. However in some cases we failed to log crucial identification data, like messageid and sender even though those were available. This is now added.
Add additional Email Signature Server tests
To make the system more reliable we added additional unit tests to verify both the flow and the insertion of email signatures.
Change configuration values to be explicit
Changed configuration variables to more accurately reflect what they are being used for, and removed hardcoded variables in favor of configuration settings.
Cleanup obsolete configuration files/update relevant ones
Since Email Signature Server was originally hosted on a different orchestration framework we still had some the configurations files related to that in the solution. These are now removed and only the actual ones are kept.
Sent items not updated
In some cases, the Email Signature Server was trying to update the email in Sent folder with the signature and not succeeding. This caused delays in sending the email or email not sent at all. This hotfix fixes it.
Sending email with add-ins disabled throws exception
When signature was not inserted (in case of no email signature available or signature already inserted in Outlook by a disabled Templafy VSTO Add-in), Email Signature Server was throwing an exception blocking the email. This is now fixed
Replace default device signatures
Default signatures added by email clients like "Sent from my iPhone" and "Send from Mail for Windows 10" are now replaced based on a master list maintained by Templafy, which also contain different translated versions. In HTML emails, default signatures are only replaced if a known signature container element with the exact text is found. In all other cases, the signature is inserted after the signature container element. In plain text emails, default signatures are only replaced if the last non-whitespace only line is identical to a default signature. In all other cases, the signature is appended.
Rename environment variables
We use environment variables to store setting related to the hosting environment and office365. Two of the environment variable names related to office365 didn't really make sense so we renamed them.
Email signature appended at the bottom of the conversation convert from html and danish
When you replied to a plain text email from Outlook (Windows or Mac) and convert the plaintext message to HTML, the Email Signature Server did not know where to introduce the signature so it was appended. This fixes the bug by adding support for PlainText to HTML conversion case.
Email signature inserted at the end of the email conversation
The problem was caused because of a case that was not covered in the Outlook Desktop identification rules. The problem is solved by adding this case of email bodies containing the email addresses between quotation marks in the rule base
Email signature not applied in sent items for email with attachment only
Email with large attachments would sometimes not be found in exchange because of the delay caused by the size. We increased the delay between search iterations in order for Exchange to finish processing the message.
Add option to log all email messages
We now support a global trace configuration for enabling tracing of all emails sent through the email signature server.
Stabilize Email Signatures for Office 365
Various improvements and fixes for signature insertion and email handling.
Support RTF mails
Email Signature Server now supports RTF emails by converting them to HTML through Exchange Web Services.
No signature added when replying from outlook for windows to an appointment
Replying to an appointment from Outlook for Windows was not working because the signature was not inserted but was appended at the end of the email and this was not supported. This is now fixed.
Send email with attachments fails if sender client is apple mail for ios
Sharing files from iOS devices through Apple mail was not working because of the Apple mail bug that converts the email body that is after the file preview. This is now fixed and the behavior is the one expected (with the Apple mail bug).
Signature not added to reply without the original message
When the original message of a reply/forward mail was deleted, the signature was not inserted at all. This is now fixed by appending the signature in case insertion matches.
Email signature not added when starting a reply email in outlook for windows and completing it in outlook online
Email signature was not added when starting a reply email in the outlook for windows and completing it in outlook online (IE browser). This is fixed now.
Ensure signed and encrypted emails (S/MIME) can be sent
Email signature server now avoids adding signature to signed and encrypted emails, this prevents corrupting the emails. It also removed DKIM signatures before adding email signatures, since Exchange will re-sign the mail with DKIM.
Don't throw exception when signature is not found
A small change in behavior, in case for some reason a user has configured his email signature, but it wasn't created and deployed correctly we no longer throw an exception, just create a log entry and continue.
Signature not added if email contains an .ics file and is sent outside the organization
Previously email signature were not added for mail with attached calendar events (.ics files), this is fixed now.
Investigate behavior when sending mail to groups and distribution lists
When sending to groups or distribution lists exchange generates automatic email for each of the user in the group, signature was not appended to these emails previously. It is fixed now.
Ensure Email Signatures are not added twice with add-ins disabled
Website: Email signatures generated by the Office add-ins did not have any identification, making it impossible for the email signature server to identify, whether they were already inserted, even with the add-ins disabled. This affected some users, where add-ins were disabled, thus getting the same signature applied twice. The email signatures generated by Templafy now contain an identifier, which can be found by the email signature server, instructing the process to skip the email from further processing. Email Signatures for Office 365: Emails sent with a signature generated by Templafy are now ignored from further processing.
Wrong conversion for special characters when replying to internal emails
Emails sent with a different encoding than UTF-8 would get destroyed when inserting the email signature. This has now been fixed and all email messages are now converted to UTF-8 to ensure compatibility with all character sets.
Sending meeting invites as TNEF gets converted to emails
Certain meeting invites didn't get correctly identified, when sent through the email signature server. This was caused by the fact that some mail clients sends meeting invite in TNEF, which was not previously encountered.
Email Signature for Office 365 is not added for internal mails
When sending emails within the same organization, messages would not get an email signature applied. We have implemented support for this scenario and are now converting the message to a format where we can insert the email signature correctly.