Wednesday, August 28, 2019

Working with dynamic inventories in Ansible using PHP (part 2)


Following the discussion in part 1, here follows the PHP code that will return the json structure required by Ansible.  It's quite complex and probably there are other better ways to do it, but that's my way.. It is optimized so that only one loop over each table will populate the structure as needed.

The database behind the scenes is PostgreSQL and you can see the structure below. Don't focus on the exact SQL query strings, as they are based on my specific data model. Depending on your model you will have to write your own queries. In the end, irrespectively of your data model,  you need two table structures, one with Group/Level associations and another with Group/Hostname associations. In my case I don't need to include any 'vars' section, so it's not shown below.

Concerning the group membership table, it's quite straight forward. The group table however, requires some special attention. You need to carefully set the grouplevel of each group, keeping in mind the 'modulo 100' and 'modulo' 20 rules. Each group with grouplevel multiple of 100 will be a major group. Each group with grouplevel multiple of 20 will be a subgroup within the major group. It may sound complex, but you actually do it once and you don't need to change frequently.

Group table
groupnamegrouplevel
south               
100
crete               
103
athens              
120
lab                 
121
islands              
140
mikonos              
141
rodos              
142
north               
200
thessaloniki        
201
os                  
300
ios                 
301
iosxr               
302
nxos                
303
junos               
304
function            
700
metro               
702
core                
703
datacenter          
720
spine               
721
leaf                
722

Group membership table
groupnamehostname
crete               
switch2              
lab                 
router1        
thessaloniki        
switch1              
ios                 
router1         
nxos                
switch1              
metro               
router1        
spine               
switch1              












Below you can see the exact PHP code along with some comments to help you understand how it works. I have tried to remove some non-critical parts to make the code more readable. Normally you should do some error checking on several parts..

<?php
  // Create connection to database  $conn pg_connect($conn_string);

  // Query the group table that allows building of groups and children  
  $sql "SELECT groupname, grouplevel FROM YOUR_TABLE ORDER BY grouplevel ASC";
  $groups pg_query($sql);

  // Query the group membership table that allows building of hosts in groups
  $sql "SELECT groupname, hostname FROM YOUR_TABLE ORDER BY grouplevel ASC";
  $groupmembers pg_query($sql);

  // Set some helper variables
  $response = array();
  $parentgroup "";
  $subgroup ""; 
  $subgrouplevel 0;

  // We loop once over the group list. We create the respective arrays and 
  // identify the children of each group based on the grouplevel hierarchy.
  // The hierarchy is based on a modulo 100 function for major groups and 
  // modulo 20 for subgroups. Very important to keep in mind that the group
  // list is sorted based on grouplevel.
  // To avoid conflicts with subgroups we restrict the subgroup range to +20 from 
  // the subgroup level 
  // Example of the hierarchy we achieve for groups residing in the range 200-299: 
  // 200 ( 201, 202, 203, 220, 240), 220 ( 221, 222), 240 ( 241, 242) 

  while ($row pg_fetch_array($groupsnullPGSQL_ASSOC)) {
      $group trim($row['groupname']); 
      $level trim($row['grouplevel']);
      if (!array_key_exists($group,$response)) $response[$group] = array();
      // if true we have identified a major group. just store the name 
      if ($level 100 == 0$parentgroup $group;
      else 
      {
        // if true we have identified a subgroup. store the name and the level and continue
        if ($level 20 == 0) {$subgroup $group$subgrouplevel $level;}

        // at the next loop if we are within the subgroup limits 
        // we set current group as subgroup child
        if ($level $subgrouplevel && $level < ($subgrouplevel 20)) 
        {
          if (!array_key_exists('children',$response[$subgroup])) $response[$subgroup]['children'] = array();  
          array_push($response[$subgroup]['children'],$group);
        }
        else // otherwise we set current group as parentgroup child
        {
          if (!array_key_exists('children',$response[$parentgroup])) $response[$parentgroup]['children'] = array();  
          array_push($response[$parentgroup]['children'],$group);
        }
      }
  }
  // We have finished setting children for each group. Time to deal with hosts
  // We loop over the group membership list. We identify the hosts and set them 
  // to their respective groups
  while ($row pg_fetch_array($groupmembersnullPGSQL_ASSOC)) {
    $group trim($row['groupname']); 
    $host trim($row['hostname']);
    if (!array_key_exists('hosts',$response[$group])) $response[$group]['hosts'] = array();
    array_push($response[$group]['hosts'],$host);
  }
  // release resources and close connection to database
  pg_free_result($groupmembers);
  pg_free_result($groups);
  pg_close($conn);
  
  // encode the array as json send it back
  echo json_encode($response);

?>

Executing the code you would get something like the following json

{"south":{"children":["crete","athens","islands"]},"crete":{"hosts":["switch2"]},"athens":{"children":["lab"]},"lab":{"hosts":["router1"]},"islands":{"children":["mikonos","rodos"]},"mikonos":[],"rodos":[],"north":{"children":["thessaloniki"]},"thessaloniki":{"hosts":["switch1"]},"os":{"children":["ios","iosxr","nxos","junos"]},"ios":{"hosts":["router1"]},"iosxr":[],"nxos":{"hosts":["switch1"]},"junos":[],"function":{"children":["metro","core","datacenter"]},"metro":{"hosts":["router1"]},"core":[],"datacenter":{"children":["spine","leaf"]},"spine":{"hosts":["switch1"]},"leaf":[]}

This json structure is acceptable by Ansible and works quite well as you can see below. I use the dynamic inventory and ask Ansible to return the groups associated with a specific host.

ansible-playbook -i ./get_inventory.php --limit router1 get_host_groups.yml 

PLAY [all] *********************************************************************************************************

TASK [show group associations for the host(s)] *********************************************************************************************************
ok: [router1 -> localhost] => 
  msg:
  - athens
  - function
  - ios
  - lab
  - metro
  - os
  - south

PLAY RECAP **********************************************************************************************************
router1                    : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

As you see, inheritance works well and although I have set only 'lab', 'ios' and 'metro' groups for my host, it is also associated with the parent groups, not explicitly specified in the membership table.

No comments:

Post a Comment