Amit Upreti
Sections
- What is XPATH?
- What is the importance of it in automation?
- What are the types of XPATH?
- How to write an XPATH?
- XPATH functions
- What is the right platform to write and verify XPATH?
- Conclusion
What is XPath?
XPath stands for XML Path Language. It is a query language used to navigate and select elements in XML documents or HTML pages. Xpath is the address of an element. XPath provides a way to locate specific elements or nodes within a structured document using path expressions and can be used in various programming languages, including Javascript, Java, XML schema, Python with scrapy framework and others.
XPath expressions are written to specify a path to the desired elements or nodes in an XML document or HTML page. These expressions can include element names, attributes, and various operators to define the specific location.
What is the importance of Xpath in automation?
XPath plays a crucial role in automation, particularly in web scraping and automated testing. It is important because it enables automation scripts to precisely locate elements within a webpage or XML document. By using XPath expressions, scripts can identify and interact with specific elements based on their structure, attributes, or content. This allows for targeted data extraction during web scraping, where XPath is used to navigate through the document’s structure and extract desired information such as text, URLs, images, or structured data. XPath is also instrumental in handling dynamic elements on web pages, as it can adapt to changes by dynamically locating elements based on their properties or relationships with other elements. In the context of automated testing, XPath is extensively used in frameworks like Selenium to locate and interact with elements during test execution, simplifying the identification of elements for testing purposes. Furthermore, XPath is a standardized language supported by various programming languages and tools, ensuring cross-platform compatibility and making it a versatile choice for automation tasks. In summary, XPath’s ability to locate elements accurately, extract data, handle dynamic elements, and offer cross-platform compatibility makes it an invaluable tool in automation workflows.
What are the types of XPath?
There are two main types of XPath expressions:
1. Absolute Xpath:
An absolute XPath expression specifies the complete path from the root element to the target element. It begins with a forward slash (/) to denote the root element and then traverses through the hierarchy of elements, specifying the element names and their positions along the path. Absolute XPath expressions are less flexible and more prone to breaking if the structure of the document changes.
Example of an absolute XPath expression:
/html/body/div[1]/div[2]/form/input[3]
2. Relative Xpath:
A relative XPath expression selects elements based on their relationship with other elements in the document. It starts with a reference point, usually with a double forward slash (//) to select elements from anywhere in the document, not just from the root. Relative XPath expressions provide more flexibility as they can adapt to changes in the document structure.
Example of a relative XPath expression:
//input[@id=’username’]
In the above example, the XPath expression selects the input element with the attribute id equal to “username” from anywhere in the document.
How to write an XPath?
To write an XPath expression, identify the target element by specifying its name or attributes. Choose between absolute (root-based) or relative (anywhere in the document) XPath. Consider using axes to navigate relationships and predicates for conditions. Test and validate the expression using appropriate tools.
Single Forward Slash:
A single forward slash (/) in an XPath expression is used to indicate the root element of the document. It is typically used at the beginning of an absolute XPath expression to specify the path starting from the root element to the target element. For example, “/html/body/div” selects the “div” element that is a direct child of the “body” element, which is in turn a direct child of the root “html” element. The single forward slash is essential for denoting the absolute path from the root element.
Double Forward Slash:
A double forward slash (//) in an XPath expression is used to select elements from anywhere in the document, regardless of their position or level within the hierarchy. It allows for a more flexible and adaptable selection approach compared to an absolute XPath. For example, “//div” selects all “div” elements in the document, regardless of their parent elements or depth in the hierarchy. The double forward slash is particularly useful for writing relative XPath expressions, as it enables the selection of elements based on their attributes or content without explicitly specifying their exact location in the document structure.
Xpath Functions:
XPath provides a variety of functions that can be used within XPath expressions to perform operations, manipulate data, or extract specific information from XML documents or HTML pages. Here are some commonly used XPath functions:
1. text(): Retrieves the text content of an element.
Example: //p/text() selects the text content of all <p> elements.
2. @attributeName: Retrieves the value of a specific attribute.
Example: //div/@class selects the value of the “class” attribute of all <div> elements.
3. contains(string1, string2): Checks if string2 is a substring of string1.
Example: //h2[contains(text(), ‘example’)] selects all <h2> elements that contain the word “example” in their text content.
4. starts-with(string1, string2): Checks if string1 starts with string2.
Example: //a[starts-with(@href, ‘https://’)] selects all <a> elements whose “href” attribute starts with “https://”.
5. position(): Returns the position of the current element within the selection.
Example: (//li)[position() = 1] selects the first <li> element among all <li> elements.
6. last(): Returns the position of the last element within the selection.
Example: (//tr/td)[last()] selects the last <td> element among all <td> elements inside <tr> elements.
These are just a few examples of XPath functions, and there are many more available, including mathematical functions, string manipulation functions, and functions for working with dates and times. XPath functions provide additional capabilities to manipulate and extract data, making XPath a powerful tool for navigating and querying XML documents or HTML pages.
Relative xpath using axes:
Relative XPath expressions can be further enhanced by utilizing axes to navigate through the document structure and specify the relationship between elements. Here are some commonly used axes in relative XPath expressions:
1. child::: Selects direct child elements of the current context node.
Example: //div/child::p selects all <p> elements that are direct children of <div> elements.
2. parent::: Selects the parent element of the current context node.
Example: //p/parent::div selects the <div> element that is the parent of <p> elements.
3. descendant::: Selects all descendant elements of the current context node, regardless of their level.
Example: //div/descendant::span selects all <span> elements that are descendants of <div> elements.
4. ancestor::: Selects all ancestor elements of the current context node, up to the root element.
Example: //span/ancestor::div selects all <div> elements that are ancestors of <span> elements.
5. following-sibling::: Selects sibling elements that appear after the current context node.
Example: //div/following-sibling::p selects all <p> elements that are siblings of <div> elements and appear after them.
6. preceding-sibling::: Selects sibling elements that appear before the current context node.
Example: //p/preceding-sibling::span selects all <span> elements that are siblings of <p> elements and appear before them.
By combining these axes with element names, attributes, or other predicates, you can construct more specific and targeted relative XPath expressions to navigate and select elements based on their relationships within the document structure.
Relative XPath without using axes
If you want to select elements using a relative XPath-like expression without using XPath itself, you can consider using alternative methods provided by the programming language or automation tool you are using. Here are a few approaches:
CSS selectors: Many programming languages and automation frameworks support CSS selectors, which are often simpler and more concise than XPath expressions. CSS selectors can be used to select elements based on their tag name, class, ID, attributes, or other properties.
DOM traversal methods: Programming languages often provide methods to navigate the Document Object Model (DOM) of a webpage. You can use these methods to traverse the DOM tree and select elements based on their relationships with other elements, such as parent-child or sibling relationships.
Element properties and attributes: Some automation tools and frameworks offer direct access to element properties and attributes. You can leverage these properties, such as class names, IDs, or custom attributes, to select elements without explicitly using XPath expressions.
While these approaches might not replicate the full flexibility and power of XPath, they can serve as alternatives for relative element selection in automation or web scraping tasks.
What is the right platform to write and verify Xpath
Using SelectorsHub:
SelectorHub is a browser extension and web scraping tool that simplifies the process of selecting and extracting data from web pages. It allows users to visually select elements on a webpage and generates corresponding CSS selectors or XPath expressions. SelectorHub provides a user-friendly interface for building selectors and supports advanced features like attribute extraction, parent-child relationships, and handling dynamic web content. It is commonly used for web scraping, data extraction, and automating web interactions. SelectorHub is available as a browser extension for Chrome and Firefox.
Conclusion
XPath is important in automation for precise element selection in web scraping and automated testing. It allows scripts to navigate XML documents or HTML pages and extract data based on element structure, attributes, or content. XPath offers flexibility and handles dynamic elements. SelectorHub is a recommended platform for writing and verifying XPath expressions, but alternatives like CSS selectors and DOM traversal methods are also available.
You can also check out this tutorial to learn more about XPath.
Read How To Scrape IMDB Movies Using Python & Scrapy using XPath selectors.
