How has Full-Text Search enhanced Business Transactions searching?

Introduction

• Atomic Scope is a functional end-to-end business activity tracking and monitoring product for Hybrid integration scenarios involving Microsoft BizTalk Server and Azure Logic Apps. With Atomic Scope, get full visibility of your end-to-end business.
• Business Transactions tracked in Atomic Scope will be displayed in the Tracking overview with tracked data. These transaction data can be searched using date-time filters, Max Records, and static filters. We can also import, export, and save these searches.
• In this blog, we will see how Full-Text search resolves the limitation in existing search functionality & made the Business Transactions searching efficient.

Key Takeaways

• Why did we build Full-Text Search?
• How to use Full-Text Search?
• An efficient way of searching Business Transactions using Full-Text Search
• Various functionality options in Full-Text Search
• Know Pros and Limitations of Full-Text Search

Why do we need a Full-Text Search?

• Before looking into Full-Text Search, let us see the existing Search functionality & what its challenges are.
• Existing search functionality
  • To search transactions in Atomic Scope, the user needs to configure the Message Content or Context properties as Global and stage properties in Business Process configuration. Those configured properties will be used as auto-popup suggestions while creating a search in the Tracking screen.
   
  • After running the searches, the global properties in the transactions will be visible in the Tracking result grid as an individual column with values, and stage property data will be visible in the Message Flow details blade under respective stages. Also, the whole properties with their tracked data will be displayed in a dedicated tab called Tracked Data in the Message Flow Details blade
   
• Challenges in Configuring properties in Business process
  • Configuring Message Content or Context properties as Global and Stage properties in Business processes is a tedious task if the transactions have dynamic properties. Whenever a new property arrives in the Message Content or context, the user must add that property in the configuration if that transaction needs to be tracked. This makes the searching process challenging.
  • Also, this requires a lot of effort, time, and accuracy. If there are any inaccuracies in the property configuration, the property data will not be tracked & it will not be displayed on the Tracking overview screen.
• How full-Text search solve this problem?
  • So, to address the above-mentioned challenges, we have added a new option in the Tracking overview screen called Full-Text Search. Using this option, you can search for any transactions in the Atomic Scope Database without configuring any properties in the business process. All you need to do is to create a search with your desired property name and value, the search engine will accurately filter your results and show them on the Tracking screen. This makes the Business Transactions search more efficient than the previous search.

Pre-requisites

• Message Archive: Full-Text Search is dependent on searching the query in Message Content or Context. Archiving the Message Content and Context for the stages in Atomic Scope is necessary. So, the user needs to configure Archive options in Business Process.
• Message Compression: Also, if you worry about the database size, currently from this version on we are storing only the compressed Message Content and Context in a database. We’re using GZipStream Microsoft class to Compress and Decompress data which follows L77 and Huffman algorithm for lossless data compression. Hence our Compression process reduces the database size significantly than before. Also running Purging Service at Regular intervals will maintain a healthy database size.

Full-Text Search

• Full Text Search Query Mechanism
  • Full-Text Search is an option in Tracking search to search the Transaction using Search Query with Message content or context properties as search keywords. While using it we do not need to configure the properties in the business process beforehand. This helps in searching for Transaction with dynamic properties.
  • You can enable the Full-Text Search switch in the Tracking screen, this will perform a Full-Text search in the Search engine and produce accurate results. Also, you must specify the Search destination which are Message Content or Message Context. You can select both options, if both are required for the search.
• Search Query and Search Destination
  • Full-text search for the keywords mentioned in the query in the Message Content or Context. Search keywords can be Message Properties or value them depending on the user’s requirements. Comparison with Property and Property value is not applicable, which means using a query like Country = India is not applicable. Instead, the user can use Country and India. So, this query will check for both the keywords in the selected Search destination content.
• Grouping and Logical operations
  • Search keywords can be constructed as joined queries using Grouping operators such as Parenthesis brackets and Logical operators (AND, OR, NOT). Here is a sample query that will be searched in Message Context.
    • Eg: ((ReceivePortName = EuropeReceiveMsgPort) OR (ReceivePortName = UKReceiveMsgPort)) AND (SenderId = SUKPackagingIndustry)
• Wild card operations
  • Also, if the user is not sure about the search keyword characters, they can use wildcard operators such as ? And *.
    • ? – denotes a single character
    • * – denotes multiple characters
    • These wildcard operators search for dynamic keywords instead of a full matching word like Send* search for SenderName or SenderId or SenderCity. Also, Te?t search for Text or Test. This wild card operation gives the flexibility to write efficient searches with fewer restrictions.
    

Note:

• Logical operators (AND, OR, NOT) are case sensitive (Uppercase), Whereas Search keywords are case Insensitive.
• Do not use double quotes or other special characters with search keywords unless you have the same characters in the search content.

Quick Overview of Full Text Search

Full-Text Search – Search Samples

• Message Content Search
  • ( (SenderId AND Contoso) AND (TransactionType AND PurchaseOrder) AND (AskPrice AND 10.4))
  • This search is used to search for the transactions that contain Message Content property and values for Sender is Contoso and TransactionType is PurchaseOrder and AskPrice is 10.4
• Message Context Search
  • (PortName AND EuropeMsgRcvPort) AND (InboundTransportType AND FILE)
  • This search is used to search for the transactions that have Message Context property and values for PortName is EuropeMsgRcvPort and InboundTransportType is FILE
• Message Context search with wildcard operations
  • (Sender* AND Con*) AND (City AND City?)
  • This search is used to search for Search content with property and values containing such as SenderId/SenderName/SenderOrg is Contoso/Connie/Connor and City is CityA/CityB/City7
• Full Text Search help info blade & feature document navigation
  • Also, there is a helpful information blade in the Tracking overview screen to guide the user with short explanations. If you need more information regarding Full Text Search user can click the hyperlink in the blade to navigate to the Full-Text Search document to know more.
  
  • These are sample searches in Full-Text Search, Users can construct the search based on their business requirements. But those searches should follow the Full-Text search syntax to avoid search parse errors and for efficient searches.

Filters, Save, Import & Export options for Full-Text Search

• Filters
  • Like the existing search, this Full-Text search can be used with Filters like Max Records, Date Time, and Static Filters.
  
• Full-Text Search Query Filter Flow
  • Date Time Filter -> Static Filters -> Max Records -> Full Text Search query -> Tracking Results
• Save, Import & Export options
  • All the existing options like Save, Import, and Export are applicable for the Full-Text Search option too.
  

Note:

• It is advisable to use the Max Records less than or equal to 1000 & set a small date time range window while using Full-Text Search options to avoid performance issues and to fetch accurate Tracking results.

 Full-Text Search – Pros and Limitations

• Pros:
  • Flexibility in creating Search queries with case insensitive search keywords
  • Option to Group the search keywords with parenthesis brackets and perform Logical operations using Logical operators such as AND, OR,
  • Possibility to perform Wild card operations with operators (* and ?) to search dynamic properties and values.
  • All filters such as Date Time Range, Max Records, and Static filters can be combined with Full-Text Search to get accurate Tracking results.
  • Allows you to search the results with less time.
  • The search bar will provide Logical operator names as intellisence in the Tracking screen to avoid Logical operator-related search parse errors.
• Limitations:
  • Comparison operation is not applicable in Full-Text Search
    • Creating a Search query with Comparison operations is not applicable in a Full-Text Search query. A search query like “ReceivePortName = EuropeMsgRcvPort” is not applicable, whereas a query like “ReceivePortName AND EuropeMsgRcvPort” is possible and this search for the availability of the two mentioned keywords in the selected Search Destination
  • Search performance
    • Search performance is dependent on Machine configuration and applied filters like DateTime, MaxRecords, and Static filters.
  • Logical operators
    • Logical operators are case sensitive (Upper case), so users must create queries with proper casing to avoid search parse errors.

Conclusion

In this blog, we have seen how Full-Text Search resolves existing Search functionality limitations and more advantages in terms of Flexibility in query construction and searching mechanism. Also, we have a lot of other new features & improvements joining the V8.3 club. To explore the full potential of the features in Atomic Scope, You can request a demo or sign up for a free trial.