1、ArcSight CorrelationFabian LibeauSuperpan 翻譯hongliangpanQQ:28797575ArcSight ESMArcSight ESM作為一款應對安全風險、合規要求和內部威脅的企業安全管理系統，ArcSight ESM（Enterprise Security Management）能夠集中展示企業信息安全各方面的概況，同時還提供有實時監視和事件關聯、風險分析、深入調查功能、報告、通知以及其他安全管理功能，可在企業范圍內全面管理、審計安全事務。 2005 ArcSight Confidential2ArcSight ESM強大的事件收集能力和跨設備

2、的事件分類能力ArcSight ESM實現了實時數據格式標準化，超過260種默認支持的設備，對每一種事件都進行了詳盡的分類，以幫助管理員理解事件的含義，并進行跨設備的分析。最為智能和靈活的關聯分析ArcSight ESM提供實時的、內存內(In-Memory) 關聯分析，具有106種預置關聯規則，圖形化規則編輯，支持資產分類、漏洞狀態與企業策略與風險管理目標的關聯。直觀的調查分析和合規性報表ArcSight ESM具有169個可重用、圖形化數據監視模塊，自由定義的儀表板（預置41個），靈活的報表格式，提供圖形化報表編輯器，提供預先打包的合規解決方案。 2005 ArcSight Confide

3、ntial3ArcSight ESM完善的自動安全響應能力ArcSight ESM可與安全設備共同協作來關閉威脅通信，以阻止正在進行的攻擊，提供威脅升級和工單處理功能。智能存儲ArcSight ESM集成了數據監控、備份腳本、分區管理等等一系列的數據庫維護工具，提供綜合安全生命周期信息管理（SLIM）策略，利用自動的高度壓縮、存檔和恢復系統以減少存儲長期安全事件所需費用。 2005 ArcSight Confidential4ArcSight ESM 2005 ArcSight Confidential5SOC中日志關聯分析的核心技術SIM/SEM/SIEM/SOC的日志關聯分析核心技術主要集

4、中在：日志收集、格式化、事件映射、關聯四個方面。日志收集：一個SIM產品是否有優勢，就要看日志收集能否支持更多的設備日志類型，能否容易擴展，自動識別支持未知設備日志。例如需要支持的協議有syslog、snmp trap、windows log、checkpoint opsec、database、file、xml、soap等等。格式化：日志收集來了，需要格式化統一標準，為后面的關聯，事件映射做準備，如果格式化不夠標準，后面也不好做。 事件映射：將日志需要統一映射成一個標準，提供統一的解決方案，這個難度也比較大，各個廠家設備的日志名稱，類型，含義都不相同，如果統一映射，是個難題。 關聯分析：這個是

5、SIM的核心部分，例如ArcSight提供了簡單的事件關聯、上下文關聯、攻擊場景關聯、低慢攻擊關聯、位置關聯、身份關聯、角色關聯等等。關聯分析還有脆弱性信息關聯、因果關聯、推理關聯等等。 關鍵問題是如何利用這些技術，給用戶提供一個很好的SIM/SEM/SIEM/SOC系統，也是一個難題。 2005 ArcSight Confidential6 2005 ArcSight Confidential7AgendaArchitectural Overview概述ArcSight Risk Prioritization風險的優先順序ArcSight different ways of correlat

6、ing information不同的關聯分析方法Rule based correlation基于規則Statistical correlation統計相關性分析Pattern discovery (advanced predictive DataMining)模式發現（先進的預測數據挖掘）ArcSight Key Concepts 2005 ArcSight Confidential8VulnerabilityAssessment漏洞評估Architectural Overview架構概述ConsoleDatabaseArcSightManagerAsset Management資產管理XML

7、Windows SystemsUnix/Linux/AIX/SolarisSecurityDevice安全設備SecurityDeviceDatabaseManagementSystemsSyslogConcentrator集中器Mainframe& Apps主機和應用SecurityDeviceData Flows數據流 2005 ArcSight Confidential9ArcSight SmartAgent Overview智能代理Largest number of supported devices 150+100% Data CaptureIntelligent Event Cap

8、ture智能事件捕獲Normalization One format規范化 - 統一格式化Categorization Grouping similar events分類 - 分組類似事件Aggregation Event redundancy (50-80% for firewalls and routers)聚集 - 事件冗余（50-80的防火墻和路由器）Filtering Transfer and store only what you need過濾轉移和存儲您所需要的Secure, configurable and governed安全，配置和管轄的FlexAgents new Sma

9、rtAgents in hours在幾個小時FlexAgents 新CounterAct Agents automated remediation抵制代理 - 自動修復Flexible Data Collection Centralized or Distributed靈活的數據收集 - 集中式或分布式Flexible Collection靈活采集CounterActSmartAgentFlexAgent 2005 ArcSight Confidential10ArcSight SmartAgent - Event Normalization and Categorization事件規范化和分

10、類Jun 01 2005 00:00:12: %PIX-3-106011: Deny inbound (No xlate) udp src outside:7/6346 dst outside:54/6346Jun 01 2005 00:00:12: %PIX-6-305011: Built dynamic TCP translation from isp:1/1967 to outside:54/62013Jun 01 2005 00:00:12: %PIX-6-302013: Built outbound TCP connection 2044303174 for outside:7/80

11、 (7/80) to isp:1/1967 (54/62013)Jun 02 2005 12:16:03: %PIX-6-106015: Deny TCP (no connection) from 02/15605 to 6/443 flags FIN ACK on interface outsideSample Raw Pix Events:Jun 02 2005 12:16:03: %PIX-6-106015: Deny TCP (no connection) from 02/15605 to 6/443 flags FIN ACK on interface outsideArcsight

12、 Categorization:Arcsight Normalization: 2005 ArcSight Confidential11ArcSight SmartAgent Guaranteed Delivery智能代理保證交付AnalystArcSightManagerPort 8443Cache緩存FailoverManager(optional)故障轉移管理器（可選）ArcSightEventArcSightEventCompressedEventSSLContentUpdates 2005 ArcSight Confidential12The ArcSight Manager - O

13、verviewReal-Time, In-Memory Correlation實時內存關聯Real-time Dashboards實時儀表盤Anomaly Detection異常檢測Correlation Rules - known behaviors關聯規則 已知行為Pattern Discovery undiscovered patterns模式發現 -未被發現的模式Flow Rates deviations from the norm流量速率-標準差 基線偏差Asset Linkage資產聯動Priority Scoring優先評分Vulnerability漏洞Asset Value資產

14、價值Severity嚴重性Alerts, among other configurable actions其他配置的行動警告Scalability and High Availability Options可擴展性和高可用性選項Intelligent Processing智能處理ManagerLINUX, Windows,UNIX, Macintosh 2005 ArcSight Confidential13AgendaArchitectural OverviewArcSight Risk Prioritization風險的優先ArcSight different ways of correl

15、ating informationRule based correlationStatistical correlationPattern discovery (advanced predictive DataMining)ArcSight Key Concepts 2005 ArcSight Confidential14ArcSight Risk Correlation風險相關性EventsScansCorrelationDevicesPrioritizationWhats happening?Whatstargeted?Whatmatters?Whats vulnerable?漏洞、 脆弱

16、= False Alarm or Normal虛假報警或普通事件= Prioritized Red Alarm優先紅色警報Dynamic Threat Severity Index動態威脅的嚴重程度指數Profiled Asset異常資產Confirmed Vulnerability已確認的漏洞Weighting Algorithms加權算法+Detected Event檢測事件ArcSight fuses all key event sources and related inputs to rank event significance on multiple variables 所有關鍵

17、的事件源和多變量等級事件 2005 ArcSight Confidential15Asset Linkage and Priority Scoring - Overview資產聯動和優先評分 - 概述Windows SystemsUnix/Linux/AIX/SolarisSecurityDeviceSecurityDeviceMainframe& AppsSecurityDevicePrioritization and Imported Scanned Assets資產的優先順序和導入掃描的資產SmartAgentsArcSightEventArcSightManagerTMArcSight

18、 Prioritized Event事件優先權VulnerabilityScanner漏洞掃描SmartAgentsAsset Information建模的程度（信心）Model ConfidenceHas asset been scanned for open ports and vulnerabilities?關聯RelevanceAre ports open on asset? Is it vulnerable?Severity嚴重性Is there a history withthis attacker or target (active lists)?資產重要性Asset Criti

19、calityHow important is thisasset to the business?代理嚴重性Agent SeverityMapping of reportingdevice severity toArcSight severity 2005 ArcSight Confidential16Asset Linkage and Priority Scoring Information Flow資產聯動和優先評分 - 信息流Vulnerability Assessment漏洞評估 Three dimensional correlation of assets, events and v

20、ulnerabilities Allows organizations to apply SIM to risk management Minimizes dead end investigations Information seamlessly linked within the ArcSight system三二維相關的資產，事件和漏洞允許企業申請SIM卡風險管理最大限度地減少死胡同調查無縫鏈接的信息系統內的ArcSightArcSight ManagerAssets Compliance Requirement Business RoleApplicationOperating Sys

21、temData roleCriticality資產重要性Vulnerabilities- Zones區ArcSightEventEvent CVEEvent Severity事件等級Priority Score Relevance 2005 ArcSight Confidential17Threat Priority Variables Considered威脅優先 多種關系組合考慮Model Confidence:How well does ArcSight know this asset?Has it been scanned?Options: 0 = Asset is not model

22、ed沒有建模 4 = Asset has not been scanned for open ports or vulnerabilities 沒有掃描端口或漏洞 8 = Asset has been scanned for open ports or vulnerabilities, but not for both掃描端口或漏洞其一10 = Asset is scanned for both open ports and vulnerabilities掃描端口和漏洞Relevance:Is the port open, and has a vulnerability been exploi

23、ted利用?Options: 5 = Assets target port is open. 5 = Event will exploit a know asset vulnerabilitySeverity:Is there a history with this attacker or target (Active Lists)?Options: 5 = Hostile List 3 = Compromised 3 = Suspicious List 1 = Reconnaissance List 5 =敵對目錄3 =不受影響 折中3 =可疑名單1 =偵察名單The Priority of

24、 an event is theAgent Severity adjusted by: Model Confidence Relevance、 Severity、 Asset Criticality一個事件的優先事項是代理嚴重性調整：模式的信心、關聯、嚴重性、資產重要性Asset Criticality:資產重要性How critical have I rated this asset within my organization.Options: 10 = Very High Criticality Assets非常高 8 = High Criticality Assets高 6 = Med

25、ium Criticality Assets中 4 = Low Criticality Assets低 2 = Very Low Criticality Assets非常低 0 = Unknown Criticality Assets未知Agent Severity:Mapping of reporting device severity to ArcSight severity.代理嚴重性：報告設備嚴重性到ArcSight的嚴重性的映射。 2005 ArcSight Confidential18Relevance drags down the Agent Severity. 相關性Examp

26、le:If Relevance = 0, the Priority = 0If Relevance = 10, the Priority = Agent SeverityModel Confidence tempers the effect of relevance on priority.建模程度Example: If Model Confidence = 0, Relevance has no effect on PriorityIf Model Confidence = 10, Priority acts the way specified above3.Formulae for the

27、 multiplication factor contributed by Model Confidence (M) and Relevance (R) R = ( R + M - R * M / 10 )If Severity (S) = 10 it adds up to 30% to Agent Severity to provide Priority: (1 + S * 3 / 100)Criticality applies a boost to Agent Severity by 20% if = (Very High)10;does nothing if Criticality =

28、(High) 8; and applies a decrement/drag if the Criticality is Medium/Low/Unknown (6/4/2): (1 + (Criticality - 8) / 10)Threat Priority The Formula威脅優先級的公式 2005 ArcSight Confidential19Heuristic: Formula-Based啟發式：按公式計算Threat level formulaPrioritizes incident investigation and responseSums up complex inf

29、ormation from the network model 威脅級別的公式事故調查和應對的優先順序匯總了網絡模型的復雜信息C:arcsightManagerconfigserverThreatLevelFormula.xml 2005 ArcSight Confidential20Priority Calculation Exercise優先級的計算練習StepsDevice Severity - Agent Severity - Calculation Exercise Agent Severity = Low Priority = 4 Asset Criticality is 0 =

30、20% decrease in priority. Priority = 3.2Severity = 0, no effect on priority. 2005 ArcSight Confidential21Priority is adjusted by Criticality通過重要性調整優先級Combined factor for model confidence and relevance, lets call it MCR = MCR is calculated using the formula R * 10 MCR = = ( R + M - R * M / 10 )where

31、R (Relavance) = 5, M (Model Confidence ) = 4MCR = 7 = 30% drop in priority again. New Priority = 3.2 * 0.7 = 2.24 rounded off gives a 2. The Final Priority is - because of low values for criticality and relevance your final priority of the event came down from 4 to 2. 2005 ArcSight Confidential22Age

32、ndaArchitectural OverviewArcSight Risk PrioritizationArcSight different ways of correlating informationRule based correlation基于規則的關聯Statistical correlationPattern discovery (advanced predictive DataMining)ArcSight Key Concepts 2005 ArcSight Confidential23Rule based correlation基于規則的關聯Fast memory base

33、d algorithm, based on RETE 2 (/rete2.htm)快速的內存算法Incorporates in Correlation:整合的相關性Events事件Vulnerability Information漏洞信息Active Lists (dynamic list with e.g. Asset/ User information)活動列表（如與動態列表資產/用戶信息Asset Categories (see later slides)資產類別（見稍后幻燈片）Asset Zones (IP ranges)資產區（IP范圍）Asset Networks (IP netw

34、orks/ groups of Asset Zones)資產網絡（IP網絡資產區/組）Results earlier rule based correlation早期規則為基礎的相關性Results earlier statistical correlation早期統計（靜態）為基礎的相關性 2005 ArcSight Confidential24Rules Theory規則理論1. Simple Aggregation Single event type or categoryBasic conditionsDe-duplication簡單 - 聚合單事件類型或類別基本條件重復數據刪除 ta

35、rgetspinge.g., any source repetitively profiling targetsarcsight_category startsWith /recontarget_address inSubnetgroupBy source_address2 or more matching events in 1 minutesource2. Complex Correlation Multi-Event JoinMultiple event types or categoriesBoolean conditionsComplete session or “round tri

36、p”復雜的關系 - 多事件加入多個事件類型或類別布爾條件完整會話或“來回”targetse.g., any source successfully engaging a targetarcsight_category startsWith /attacktarget_address inSubnet groupBy source_address, target_address1+ matching events in 1 minutejoin events across IDS, firewall, and host3. Complex Long SequenceMultiple sessio

37、nsPre-attack probes,attack formation/progression, and attack conclusionHandles long-term memory need using active lists 復雜鈥長序列多個會話、預探測攻擊，攻擊編隊/進程，攻擊結束處理長期記憶需要使用活動列表attackFWIDSe.g., low&slow attack pattern across multiple rules/recon rule records source_address suspicious/attack rule upgrades source_a

38、ddress to hostileand records target_address as compromisedFinal rule looks for evidence of successrule1activelistactivelistrule2rule3sourceRule Types By Complexity復雜規則類型Example例子Approach方法途徑Catch and accumulate events in real-time in memory- - - - - - - -Good for event bursts在內存中捕獲和累積事件良好的突發事件Catch

39、and correlate events in real-time in memory until the rule chain is complete- - - - - - - -Good for cross-event matching that occurs in a single session在內存中捕獲和累積事件直到完成該規則鏈 - - - - - - - - 良好的交叉配對活動，在單個會話發生Break up sequences in logical segments and maintain active lists in the database that tie toget

40、her multiple rules - - - - - - - -Good for long elapsed time attack sequences that start and stop across multiple sessions打破序列邏輯段，保持積極的數據庫列出了多個規則聯系在一起 - - - - 經過好長的時間序列，開始攻擊和跨多個會話停止 2005 ArcSight Confidential25Simple Correlation: Event Aggregation簡單的相關性：事件聚集Most basic correlation最基礎的關聯De-duplicates

41、events (many-to-one)去重Single source, single target單一源單一目標Flatten event bursts壓扁事件爆發ArcSight SmartAgents do this too!CorrelationSingle EventMultiple Events(same base event)As above plusDistributed attack sources分布攻擊源Multiple attack targets多攻擊目標Any field or combination of event fields (types of event)

42、人行事件領域（事件類型的組合）Interrelates diverse events不同的事件相互聯系CorrelationSingle EventMultiple Events(multiple event types, sources and/or targets) 2005 ArcSight Confidential26Simple Correlation: Event Aggregation簡單的相關性：事件聚集Most basic correlation最基礎的關聯De-duplicates events (many-to-one)去重Single source, single ta

43、rget單一源單一目標Flatten event bursts壓扁事件爆發ArcSight SmartAgents do this too!CorrelationSingle EventMultiple Events(same base event)As above plusDistributed attack sources分布攻擊源Multiple attack targets多攻擊目標Any field or combination of event fields (types of event)人行事件領域（事件類型的組合）Interrelates diverse events不同的事

44、件相互聯系CorrelationSingle EventMultiple Events(multiple event types, sources and/or targets) 2005 ArcSight Confidential27Advanced Correlation: Multi-event Joins高級的相關性：多事件加人Inter-relates (joins) diverse events with any combination of common field values e.g., source IP, target IP, port, protocol, userna

45、me, domain, location, zone etc分析不同事件的相互聯系，with事件通用屬性：例如，源IP，目標IP，端口，協議，用戶名，域，位置，區域等Compare any event fields using flexible boolean logic (AND, OR, NOT)比較任意事件字段采用比較靈活的布爾邏輯（與，或，非）Good for cross-event matching of complete end-to-end sessions良好的跨事件的完整的端至端會話匹配E.g. correlating an attacker detected by NIDS

46、, crossing the firewall, compromising a host, creating a back connection to steal confidential dataCorrelationSingle EventMultiple Events with Common Event Fields (different base events)在事件通用屬性上分析多事件 2005 ArcSight Confidential28Complex Correlation: Attack State Monitoring復雜的相關性：攻擊狀態監測Inter-relates e

47、vents across sessions using Active Lists使用活動列表分析跨多會話事件Any field or combination of event fields may be persisted from base events任何字段或字段組合的事件可能會從基本事件提煉Long & short -term state machines長期與短期的狀態機Good for tracking logical sequences of events良好的跟蹤事件的邏輯順序E.g. Reconnaissance, attack formation, progression

48、& conclusion例如偵察，攻擊形成，進展及結論CorrelationEvent Sequence 1(multi-event joins)Record on Active List(state 1)CorrelationEvent Sequence 2Event Sequence 3CorrelationRecord on Active List(state 2)Single Event 2005 ArcSight Confidential29 ( 2 ) ( 1 )Rule based Cross-Correlation基于規則的交叉關聯分析Scenario 1 The attack

49、er is unsuccessful and alarms are false positives方案1 - 攻擊不成功和報警器誤報HackerN-IDSIDS reports WEB-IIS ISAPI .printer access to 48ArcSight categorizes the signature as /Attack/ and recognizes thatthe target is hosting Mission Critical ApplicationsArcSight correlates and fires the 1st rule Yellow Alarm: /

50、Attack Started / Perimeter Alarm / Mission Critical Asset ( Warning_Display )The source IP address is quietly recorded as suspiciousFWFirewall reports a “drop” from that source IP to that target IP address ArcSight correlates and fires the 2nd and final rule Green Status: / Attack Blocked / Dropped

51、at Firewall / Mission Critical Asset ( Information_Display )The target host is never touchedArcSight records the event for an audit trail, alarms are suppressed and the source IP address remains on the suspicious list48 Host 2005 ArcSight Confidential30Hacker19 Scenario 2 The attacker is successful

52、方案2 - 攻擊者成功N-IDSIDS reports WEB-IIS ISAPI .printer access to 48ArcSight categorizes the signature as /Attack/ and recognizes thatthe target is hosting Mission Critical Applications ( 1 )ArcSight correlates and fires the 1st rule Yellow Alarm: / Attack Started / Perimeter Alarm / Mission Critical Ass

53、et ( Warning_Display )The source IP address is quietly recorded as suspiciousFWFirewall reports an “accept” from that source IP to that target IP ( 2 )ArcSight correlates and fires the 2nd rule Red Alarm: / Attack Progressing / Crossed Firewall / Mission Critical Asset ( Threat_Display )The source I

54、P address is upgraded onto the hostile list48 The target gets “back doored”, indicated when thefirewall reports an FTP back out from the target to the attack sourceArcSight looks for FTP out signatures across different devicesHost ( 3 )ArcSight correlates and fires 3nd and final rule Double Red Alar

55、m: / Attack Succeeds / Compromised Target / Mission Critical Asset ( Confirmation_Display )The Target IP address is recorded as compromised, and an automated notification is sentRule based Cross-Correlation基于規則的交叉關聯分析 2005 ArcSight Confidential31ArcSight Rule Editor規則編輯JoinCondition 1Condition 2Incl

56、ude predefined Filters/ ConditionsAdd Asset InformationAdd Vulnerability InformationAdd Information from Active List Explanation: This rule looks for a correlation event that triggers from an attack against a system and the attacked system begins attacking other systems說明：此規則尋找一個相關事件觸發從一個對一個系統的攻擊和被攻

57、擊的系統開始攻擊其它系統 2005 ArcSight Confidential32AgendaArchitectural OverviewArcSight Risk PrioritizationArcSight different ways of correlating informationRule based correlationStatistical correlation 統計關聯Pattern discovery (advanced predictive DataMining)ArcSight Key Concepts 2005 ArcSight Confidential33Arc

58、Sight Statistical Correlation統計相關性ArcSight offers the following statistic types:Moving Average、Average、Identity、Kurtosis、SkewStandard DeviationVariance統計的ArcSight提供以下類型：移動平均線、平均、身分峰度、斜、標準差、方差Alarm from those can be used for rule based correlations. 34Statistical Correlation: Event Rates統計相關性：事件發生率Ch

59、oice of statistical function moving average, standard deviation, skew, variance or Kurtosis統計功能選擇移動平均，標準偏差，偏差，方差或峰度Configurable sample period & interval配置的抽樣周期和間隔CorrelationSteady Stream of Events(same base event)源源不斷的事件流（相同的基礎事件）Controllable event frequency可控事件頻率Multiple attack dimensions多種攻擊尺度Any

60、field or combination of event fields (types of event)任意事件字段或組合字段Much more sophisticated than simply considering the rate of all events directed at a single targetCorrelationSingle EventChange in Base Event Rate(statistical function)變化中的事件發生率（統計功能）No Events更為復雜的不僅僅是考慮到在一個單一的目標指示所有事件發生率 2005 ArcSight